c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,920 Commits 171 Branches 49 Tags
15a121fec52b995b71a53706f4c132c203d00384
Commit Graph
3920 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Savaş Yıldırım 15a121fec5 Update README.md (#4315) 2020-05-12 15:01:34 -04:00
Stefan Schweter 15d45211f7 [model_cards]: 🇹🇷 Add new ELECTRA small and base models for Turkish (#4318) 2020-05-12 15:01:17 -04:00
Viktor Alm 8a017cbb5a Add modelcard with acknowledgements (#4321) 2020-05-12 15:00:56 -04:00
Julien Chaumond 4bf5042240 Fix BART tests on GPU (#4298) 2020-05-12 09:11:50 -04:00
Viktor AlmViktor AlmJulien Chaumond
 		e4512aab3b Add MultipleChoice to TFTrainer [WIP] (#4270) 
* catch gpu len 1 set to gpu0

* Add mpc to trainer

* Add MPC for TF

* fix TF automodel for MPC and add Albert

* Apply style

* Fix import

* Note to self: double check

* Make shape None, None for datasetgenerator output shapes

* Add from_pt bool which doesnt seem to work

* Original checkpoint dir

* Fix docstrings for automodel

* Update readme and apply style

* Colab should probably not be from users

* Colabs should probably not be from users

* Add colab

* Update README.md

* Update README.md

* Cleanup __intit__

* Cleanup flake8 trailing comma

* Update src/transformers/training_args_tf.py

* Update src/transformers/modeling_tf_auto.py

Co-authored-by: Viktor Alm <viktoralm@pop-os.localdomain>
Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-12 08:48:48 -04:00
Levent Serinol 65be574aec fixed missing torch module import (#4305) 
fixed missing torch module import in example usage code
 2020-05-12 08:34:17 -04:00
Jangwon Park 31e67dd19f Remove hard-coded pad token id in distilbert and albert (#3965) 2020-05-12 08:32:44 -04:00
Lysandre Debut 30e343862f pin TF to 2.1 (#4297) 
* pin TF to 2.1

* Pin flake8 as well
 2020-05-11 21:03:30 -04:00
Julien Chaumond 56e8ef632f [ci] Restrict GPU tests to actual code commits 2020-05-11 20:40:41 -04:00
Julien Chaumond ba6f6e44a8 [ci] Re-enable torch GPU tests 2020-05-12 00:05:36 +00:00
Lysandre Debut 9524956819 Documentation specification (#4294) 2020-05-11 16:43:57 -04:00
Bram Vanroy 61d22f9cc7 Simplify cache vars and allow for TRANSFORMERS_CACHE env (#4226) 
* simplify cache vars and allow for TRANSFORMERS_CACHE env

As it currently stands, "TRANSFORMERS_CACHE" is not an accepted variable. It seems that the these variables were not updated when moving from version pytorch_transformers to transformers. In addition, the fallback procedure could be improved. and simplified. Pathlib seems redundant here.

* Update file_utils.py
 2020-05-11 15:24:02 -04:00
Lysandre Debut cd40cb8879 Fix special token doc (#4292) 2020-05-11 15:05:36 -04:00
Tianlei Wu 82601f4c1a Allow gpt2 to be exported to valid ONNX (#4244) 
* allow gpt2 to be exported to valid ONNX model

* cast size from int to float explictly
 2020-05-11 14:55:55 -04:00
Guo, Quan 39994051e4 Add migrating from pytorch-transformers (#4273) 
"Migrating from pytorch-transformers to transformers" is missing in the main document. It is available in the main `readme` thought. Just move it to the document.
 2020-05-11 13:35:13 -04:00
Lysandre Debut 051dcb2a07 CamemBERT does not make use of Token Type IDs (#4289) 2020-05-11 13:31:03 -04:00
fgaim 41e8291217 Add ALBERT to the Tensorflow to Pytorch model conversion cli (#3933) 
* Add ALBERT to convert command of transformers-cli

* Document ALBERT tf to pytorch model conversion
 2020-05-11 13:10:00 -04:00
Stefan Schweter 3f42eb979f Documentation: fix links to NER examples (#4279) 
* docs: fix link to token classification (NER) example

* examples: fix links to NER scripts
 2020-05-11 12:48:21 -04:00
Funtowicz Morgan 8fdb7997c6 Align sentiment-analysis' tokenizer (currently uncased) to the model (uncased). (#4264) 2020-05-11 12:45:53 -04:00
Sam Shleifer 4658896ee1 [Marian] Fix typo in docstring (#4284) 2020-05-11 11:47:51 -04:00
Levent SerinolandJulien Chaumond bf64b8cf09 Model card for bert-turkish-question-answering question-answering model (#4281) 
* Create README.md

* Update model_cards/lserinol/bert-turkish-question-answering/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-11 11:32:25 -04:00
Julien Plu 94b57bf796 [TF 2.2 compat] use tf.VariableAggregation.ONLY_FIRST_REPLICA (#4283) 
* Fix the issue to properly run the accumulator with TF 2.2

* Apply style

* Fix training_args_tf for TF 2.2

* Fix the TF training args when only one GPU is available

* Remove the fixed version of TF in setup.py
 2020-05-11 11:28:37 -04:00
Savaş Yıldırım cffbb3d8ed Update README.md (#4276) 2020-05-11 11:24:41 -04:00
Julien Plu 5f50d619dd Fix XTREME link + add number of eval documents + fix usage code (#4280) 2020-05-11 11:24:10 -04:00
theblackcat102 7751be7cee fix reformer apex scaling issue (#4242) 2020-05-11 16:53:42 +02:00
Patrick von Platen ac7d5f67a2 [Reformer] Add Enwiki8 Reformer Model - Adapt convert script (#4282) 
* adapt convert script

* update convert script

* finish

* fix marian pretrained docs
 2020-05-11 16:38:07 +02:00
Patrick von Platen 336116d960 Reformer enwik8 - Model card (#4286) 2020-05-11 16:22:08 +02:00
flozi00 b290c32e16 [docs] fix typo (#4249) 2020-05-10 14:07:08 -04:00
Sam Shleifer 3487be75ef [Marian] documentation and AutoModel support (#4152) 
- MarianSentencepieceTokenizer - > MarianTokenizer
- Start using unk token.
- add docs page
- add better generation params to MarianConfig
- more conversion utilities
 2020-05-10 13:54:57 -04:00
Girishkumar 9d2f467bfb [README] Corrected some grammatical mistakes (#4199) 2020-05-10 09:02:36 -04:00
Julien Chaumond 7b75aa9fa5 [TPU] Doc, fix xla_spawn.py, only preprocess dataset once (#4223) 
* [TPU] Doc, fix xla_spawn.py, only preprocess dataset once

* Update examples/README.md

* [xla_spawn] Add `_mp_fn` to other Trainer scripts

* [TPU] Fix: eval dataloader was None
 2020-05-08 14:10:05 -04:00
Julien Chaumond 274d850d34 Fix #4098 2020-05-08 12:39:46 -04:00
Lorenzo De MatteiandJulien Chaumond 26dad0a9fa example updated to use generation pipeline (#4230) 
* example updated to use generation pipeline

* Update model_cards/LorenzoDeMattei/GePpeTto/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-08 09:45:10 -04:00
rmroczkowski 9ebb5b2a54 Model card for allegro/herbert-klej-cased-tokenizer-v1 (#4184) 2020-05-08 09:42:43 -04:00
rmroczkowski 9e54efd004 Model card for allegro/herbert-klej-cased-v1 (#4183) 2020-05-08 09:42:28 -04:00
Manuel Romero a8b798e6c4 Model card for spanish electra small (#4196) 2020-05-08 09:30:15 -04:00
Savaş Yıldırım 242005d762 Create README.md (#4132) 
* Create README.md

* Adding code fence around code block
 2020-05-08 09:27:29 -04:00
Manuel Romero 5940c73bbb Create README.md (#4179) 
model card for my De Novo Drug discovery model using MLM
 2020-05-08 09:25:36 -04:00
Patrick von Platen cf08830c28 [Pipeline, Generation] tf generation pipeline bug (#4217) 
* fix PR

* move tests to correct place
 2020-05-08 08:30:05 -04:00
Jared T NielsenandLysandre 8bf7312654 Add AlbertForPreTraining and TFAlbertForPreTraining models. (#4057) 
* Add AlbertForPreTraining and TFAlbertForPreTraining models.

* PyTorch conversion

* TensorFlow conversion

* style

Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2020-05-07 19:44:51 -04:00
Julien Chaumond c99fe0386b [doc] Fix broken links + remove crazy big notebook 2020-05-07 18:44:18 -04:00
Savaş Yıldırım 66113bd626 Create README.md (#4202) 2020-05-07 18:31:22 -04:00
Julien Chaumond 6669915b65 [examples] Add column for pytorch-lightning support 2020-05-07 15:26:58 -04:00
Julien Chaumond 612fa1b10b Examples readme.md (#4215) 
* README

* Update README.md
 2020-05-07 15:00:06 -04:00
Lysandre 2e57824374 Pin isort and tf <= 2.1.0 2020-05-07 14:42:00 -04:00
Lysandre e7cfc1a313 Release: v2.9.0 v2.9.0 2020-05-07 14:15:20 -04:00
Julien Chaumond 0ae96ff8a7 BIG Reorganize examples (#4213) 
* Created using Colaboratory

* [examples] reorganize files

* remove run_tpu_glue.py as superseded by TPU support in Trainer

* Bugfix: int, not tuple

* move files around
 2020-05-07 13:48:44 -04:00
Julien Chaumond cafa6a9e29 [Trainer] Ability to specify optimizer/scheduler at init 
cc @patrickvonplaten @thomwolf
 2020-05-07 11:25:26 -04:00
Bram Vanroy e4fd5e3999 Use with_extension to change the extension (#4203) 
As per https://github.com/huggingface/transformers/pull/3934#discussion_r421307659
 2020-05-07 11:14:56 -04:00
Lysandre DebutandJulien Chaumond ebf80e2e70 Tpu trainer (#4146) 
* wip

* wip

* a last wip

* Better logging when using TPUs

* Correct argument name

* Tests

* fix

* Metrics in evaluation

* Update src/transformers/training_args.py

* [tpu] Use launcher script instead

* [tpu] lots of tweaks

* Fix formatting

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-07 10:34:04 -04:00