9a3f91088c
* Add MobileBert
* Quality + Conversion script
* style
* Update src/transformers/modeling_mobilebert.py
* Links to S3
* Style
* TFMobileBert
Slight fixes to the pytorch MobileBert
Style
* MobileBertForMaskedLM (PT + TF)
* MobileBertForNextSentencePrediction (PT + TF)
* MobileFor{MultipleChoice, TokenClassification} (PT + TF)
ss
* Tests + Auto
* Doc
* Tests
* Addressing @sgugger's comments
* Adressing @patrickvonplaten's comments
* Style
* Style
* Integration test
* style
* Model card
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
language, thumbnail, license
| language | thumbnail | license |
|---|---|---|
| english | https://huggingface.co/front/thumbnails/google.png | apache-2.0 |
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks.
This checkpoint is the original MobileBert Optimized Uncased English: uncased_L-24_H-128_B-512_A-4_F-4_OPT checkpoint.
How to use MobileBERT in transformers
from transformers import pipeline
fill_mask = pipeline(
"fill-mask",
model="google/mobilebert-uncased",
tokenizer="google/mobilebert-uncased"
)
print(
fill_mask(f"HuggingFace is creating a {fill_mask.tokenizer.mask_token} that the community uses to solve NLP tasks.")
)