c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,419 Commits 171 Branches 49 Tags
08a70fb392e5fbc3229385bf131c4aed5681f331
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond 08a70fb392 [model_cards] Fixup d6df9a8f 2020-03-06 17:50:38 -05:00
Julien Chaumond 8a2d9bc9ef Add model cards for DeepPavlov models (#3138) 
* add empty model cards for every current DeepPavlov model

* fix: replace cyrillic `с` with `c`

* docs: add model cards for current DeepPavlov BERT models

* docs: add links for arXiv preprints
 2020-03-05 09:34:43 -05:00
Julien Chaumond 3f067f4409 [hf_api] slightly more doc 2020-03-04 23:55:46 -05:00
Julien Chaumond f564f93c84 [hf_api] Get the public list of all the models on huggingface 2020-03-04 23:33:09 -05:00
Julien Chaumond ff9e79ba3a make style 2020-03-04 20:18:07 -05:00
Julien Chaumond c440030e99 [model_cards] Tag AR model languages 2020-03-04 16:33:10 -05:00
Julien Chaumond 76111a3d3a [model_cards] Add card by @lvwerra 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:55:20 -05:00
Julien Chaumond a43c388abb [model_cards] Add card by @djstrong 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:53:02 -05:00
Julien Chaumond 256cbbc4a2 [doc] Fix link to how-to-train Colab 2020-03-04 12:01:45 -05:00
Julien Chaumond f631e01d2c [ci] Re-run integration ground truth from fairseq 
Adopted best practice set by @patrickvonplaten of commenting lines run on fairseq, for easy comparison

also see #3020
 2020-03-03 15:31:40 -05:00
Julien Chaumond a088d75e51 [model_cards] Fix incorrect path 2020-03-03 09:52:32 -05:00
Julien Chaumond eec5ec8071 [BART] to each its own config + make BART compatible w/ Pipelines 
cc @sshleifer
 2020-03-02 18:56:17 -05:00
Julien Chaumond f169957d0c TF GPU CI (#3085) 
* debug env

* Restrict TF GPU memory

* Fixup

* One more test

* rm debug logs

* Fixup
 2020-03-02 15:45:25 -05:00
Julien Chaumond 0e56b37e80 rm bogus file 
cc @patrickvonplaten
 2020-03-02 12:27:12 -05:00
Julien ChaumondFuntowicz MorganLysandre Debut
 		13afb71208 [ci] Ensure that TF does not preempt all GPU memory for itself 
see https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth

Co-Authored-By: Funtowicz Morgan <mfuntowicz@users.noreply.github.com>
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-03-02 11:56:45 -05:00
Julien Chaumond 298bed16a8 make style 2020-03-01 14:08:01 -05:00
Julien Chaumond d6ef587a10 [ci] Fixup e36bd94345 2020-02-28 23:19:17 -05:00
Julien Chaumond e36bd94345 [ci] Run all tests on (self-hosted) GPU (#3020) 
* Create self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* do not run slow tests, for now

* [ci] For comparison with circleci, let's also run CPU-tests

* [ci] reorganize

* clearer filenames

* [ci] Final tweaks before merging

* rm slow tests on circle ci

* Trigger CI

* On GPU this concurrency was way too high
 2020-02-28 21:11:08 -05:00
Julien Chaumond b370cc7e99 [gpu] Fixup fdd61b1992 2020-02-26 21:48:49 +00:00
Julien Chaumond f5516805c2 Fix bart slow test 2020-02-26 20:47:49 +00:00
Julien ChaumondandSam Shleifer 9cda3620b6 Fix (non-slow) tests on GPU (torch) (#3024) 
* Fix tests on GPU (torch)

* Fix bart slow tests

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-02-26 11:59:25 -05:00
Julien Chaumond e693cd1e87 [ci] Run slow tests every day 2020-02-24 19:54:47 -05:00
Julien Chaumond 4fc63151af [ci] Attempt to fix #2844 2020-02-24 19:51:34 -05:00
Julien Chaumond 73028c5df0 [model_cards] EsperBERTo 2020-02-14 15:16:33 -05:00
Julien ChaumondandIlias Chalkidis 4e69104a1f [model_cards] Also use the thumbnail as meta 
Co-Authored-By: Ilias Chalkidis <ihalk@di.uoa.gr>
 2020-02-14 10:27:11 -05:00
Julien ChaumondandYves Peirsman 73d79d42b4 [model_cards] nlptown/bert-base-multilingual-uncased-sentiment 
cc @yvespeirsman

Co-Authored-By: Yves Peirsman <yvespeirsman@users.noreply.github.com>
 2020-02-14 09:51:11 -05:00
Julien Chaumond 7d22fefd37 [pipeline] Alias NerPipeline as TokenClassificationPipeline 2020-02-14 09:18:10 -05:00
Julien Chaumond 925a13ced1 [model_cards] mv README.md 2020-02-13 23:07:29 -05:00
Julien Chaumond 4d36472b96 [run_ner] Don't crash if fine-tuning local model that doesn't end with digit 2020-02-14 03:25:29 +00:00
Julien Chaumond 21da895013 [model_cards] Better image for social sharing 2020-02-11 20:30:08 -05:00
Julien Chaumond 9a70910d47 [model_cards] Tweak @mrm8488's model card 2020-02-11 20:20:39 -05:00
Julien Chaumond 9274734a0d [model_cards] mv to correct location + tweak tag 2020-02-11 20:13:57 -05:00
Julien Chaumond e0b6247cf7 [model_cards] Change formatting slightly as we updated our markdown engine 
cc @tholor @loretoparisi @simonefrancia
 2020-02-11 18:25:21 -05:00
Julien Chaumond 1f5db9a13c [model_cards] Rm extraneous tag 2020-02-10 17:45:13 -05:00
Julien Chaumond 95bac8dabb [model_cards] Add language metadata to existing model cards 
This will enable filtering on language (amongst other tags) on the website

cc @loretoparisi, @stefan-it, @HenrykBorzymowski, @marma
 2020-02-10 17:42:42 -05:00
Julien ChaumondandKevin Canwen Xu b4691a438d [model_cards] BERT-of-Theseus: use the visual as thumbnail 
cc @jetrunner

Co-Authored-By: Kevin Canwen Xu <canwenxu@outlook.com>
 2020-02-10 11:27:08 -05:00
Julien Chaumond fc325e97cd [model_cards] Showcase model tag syntax 2020-02-10 11:27:08 -05:00
Julien Chaumond 63a5399bc4 [model_cards] Specify language meta + thumbnail 
cc @tholor

see #2799
 2020-02-10 11:20:05 -05:00
Julien Chaumond 42f08e596f [examples] rename run_lm_finetuning to run_language_modeling 2020-02-07 09:15:28 -05:00
Julien Chaumond 4f7bdb0958 [examples] Fix broken markdown 2020-02-07 09:15:28 -05:00
Julien Chaumond 7748cbbe7d Oopsie 2020-02-06 15:30:02 -05:00
Julien Chaumond 432c12521e [docs] Add menu w/ links to other pages on hf.co 2020-02-06 15:30:02 -05:00
Julien Chaumond eae8ee0389 [doc] model sharing: mention README.md + tweaks 
cc @lysandrejik @thomwolf
 2020-02-05 14:20:03 -05:00
Julien ChaumondandLysandre Debut ada24def22 [run_lm_finetuning] Tweak fix for non-long tensor, close #2728 
see 1ebfeb7946 and #2728

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-02-05 12:49:18 -05:00
Julien Chaumond 9a50828b5c Pipelines: fix crash when modelcard is None 
cc @mfuntowicz does this seem correct?
 2020-02-03 17:53:39 -05:00
Julien ChaumondandStefan Schweter ddb6f9476b [model_cards] dbmdz models 
Co-Authored-By: Stefan Schweter <stefan-it@users.noreply.github.com>
 2020-01-31 18:39:09 -05:00
Julien ChaumondandHenrykBorzymowski 6636826f04 [model_cards] Multilingual + Dutch SQuAD2.0 
Co-Authored-By: HenrykBorzymowski <henrykborzymowski@users.noreply.github.com>
 2020-01-31 18:39:09 -05:00
Julien ChaumondLoreto ParisiSimone Francia
 		98dadc98e1 [model_cards] UmBERTo 
Co-Authored-By: Loreto Parisi <loretoparisi@gmail.com>
Co-Authored-By: Simone Francia <francia.simone1@gmail.com>
 2020-01-31 18:39:09 -05:00
Julien Chaumond d6fc34b459 [model_cards] add mine 2020-01-31 18:39:09 -05:00
Julien Chaumond 5a6b138b00 [Umberto] model shortcuts (#2661) 
* [Umberto] model shortcuts

cc @loretoparisi @simonefrancia

see #2485

* Ensure that tokenizers will be correctly configured
 2020-01-30 21:05:53 -05:00
Julien Chaumond 7fe294bf07 Hotfix: same handling of non-existent files as for config 2020-01-30 20:05:04 -05:00
Julien Chaumond b85c59f997 config.architectures 2020-01-30 19:26:59 -05:00
Julien Chaumond f9bc3f5771 style tweak 2020-01-30 19:26:59 -05:00
Julien Chaumond 0b13fb822a No need for a model_type here 
cc @lysandrejik
 2020-01-30 19:26:59 -05:00
Julien Chaumond 9fa836a73f fill_mask helper (#2576) 
* fill_mask helper

* [poc] FillMaskPipeline

* Revert "[poc] FillMaskPipeline"

This reverts commit 67eeea55b0f97b46c2b828de0f4ee97d87338335.

* Revert "fill_mask helper"

This reverts commit cacc17b884e14bb6b07989110ffe884ad9e36eaa.

* README: clarify that Pipelines can also do text-classification

cf. question at the AI&ML meetup last week, @mfuntowicz

* Fix test: test feature-extraction pipeline

* Test tweaks

* Slight refactor of existing pipeline (in preparation of new FillMaskPipeline)

* Extraneous doc

* More robust way of doing this

@mfuntowicz as we don't rely on the model name anymore (see AutoConfig)

* Also add RobertaConfig as a quickfix for wrong token_type_ids

* cs

* [BIG] FillMaskPipeline
 2020-01-30 18:15:42 -05:00
Julien Chaumond 6b4c3ee234 [run_lm_finetuning] GPT2 tokenizer doesn't have a pad_token 
ping @lysandrejik
 2020-01-27 20:14:02 -05:00
Julien Chaumond 79815bf666 [serving] Fix typo 2020-01-27 19:58:25 -05:00
Julien Chaumond 5004d5af42 [serving] Update dependencies 2020-01-27 19:58:00 -05:00
Julien Chaumond ea56d305be make style 2020-01-27 12:13:32 -05:00
Julien Chaumond 11b13e94a3 Add type to help my IDE out 2020-01-24 14:00:57 -05:00
Julien Chaumond 90b7df444f Upload CLI: on win32, use slashes, not os.sep 2020-01-22 22:41:21 -05:00
Julien Chaumond 119dc50e2a Doc tweak on model sharing 2020-01-22 22:40:38 -05:00
Julien Chaumond 34a3c25a30 Fix for XLMRobertaConfig inherits from RobertaConfig 
hat/tip @stefan-it
 2020-01-22 17:50:24 -05:00
Julien Chaumond 1a8e87be4e Line-by-line text dataset (including padding) 2020-01-21 16:57:38 -05:00
Julien Chaumond b94cf7faac change order 2020-01-21 16:57:38 -05:00
Julien Chaumond 2eaa8b6e56 Easier to not support this, as it could be confusing 
cc @lysandrejik
 2020-01-21 16:57:38 -05:00
Julien Chaumond 801aaa5508 make style 2020-01-21 16:57:38 -05:00
Julien Chaumond 56d4ba8ddb [run_lm_finetuning] Train from scratch 2020-01-21 16:57:38 -05:00
Julien Chaumond 23a2cea8cb Tokenizer.from_pretrained: fetch all possible files remotely 2020-01-16 16:47:19 -05:00
Julien Chaumond 99f9243de5 same here, try to not serialize too much if unneeded 2020-01-16 16:47:19 -05:00
Julien Chaumond 9d8fd2d40e tokenizer.save_pretrained: only save file if non-empty 2020-01-16 16:47:19 -05:00
Julien Chaumond d9fa1bad72 Fix failing torchscript test for xlnet 
model.parameters() order is apparently not stable (only for xlnet, for some reason)
 2020-01-15 20:22:21 -05:00
Julien Chaumond a98b2ca8c0 Style + fixup BertJapaneseTokenizer 2020-01-15 19:05:51 -05:00
Julien Chaumond 83a41d39b3 💄 super 2020-01-15 18:33:50 -05:00
Julien Chaumond cd51893d37 Merge branch 'Rexhaif-patch-1' 2020-01-15 18:25:15 -05:00
Julien Chaumond 248aeaa842 Merge branch 'patch-1' of https://github.com/Rexhaif/transformers into Rexhaif-patch-1 2020-01-15 18:22:01 -05:00
Julien Chaumond eb59e9f705 Graduate sst-2 to a canonical one 2020-01-15 16:28:50 +00:00
Julien Chaumond e184ad13cf Close #2392 2020-01-15 15:43:44 +00:00
Julien Chaumond 715fa638a7 Merge branch 'master' into from_scratch_training 2020-01-14 18:58:21 +00:00
Julien Chaumond 764f836d52 Update test_tokenization_auto.py 2020-01-13 22:50:34 -05:00
Julien Chaumond d5831acb07 Update test_tokenization_auto.py 2020-01-13 22:47:33 -05:00
Julien Chaumond ed6cd597cc Update test_tokenization_auto.py 2020-01-13 22:46:35 -05:00
Julien Chaumond 5cb463a714 Update test_tokenization_auto.py 2020-01-13 22:38:29 -05:00
Julien Chaumond afc24ea5d4 In a parallel setup this could fail 2020-01-13 23:44:08 +00:00
Julien Chaumond 894812c652 Fixup mapping 2020-01-13 23:34:19 +00:00
Julien Chaumond b20f11d4ca 🔫 Python35 2020-01-13 23:20:44 +00:00
Julien Chaumond 0304628590 Map configs to models and tokenizers 2020-01-13 23:11:44 +00:00
Julien Chaumond 1fc855e456 [tests] Safety checks on CONFIG_MAPPING 2020-01-13 21:52:55 +00:00
Julien Chaumond 3c86b6f3c5 Py35 doesn't like inline variable types 2020-01-13 20:44:33 +00:00
Julien Chaumond b803b067bf Config to Model mapping 2020-01-13 20:05:20 +00:00
Julien Chaumond cf8a70bf68 More AutoConfig tests 2020-01-11 03:43:57 +00:00
Julien Chaumond 6bb3edc300 Serialize model_type if exists 2020-01-11 03:18:56 +00:00
Julien Chaumond c6f682c1eb flake 2020-01-11 03:18:31 +00:00
Julien Chaumond 4d1c98c012 AutoConfig + other Auto classes honor model_type 2020-01-11 02:46:17 +00:00
Julien Chaumond 2f32dfd33b Convention: name mixins mixins 2020-01-11 01:24:29 +00:00
Julien Chaumond 055e80cfad rm old ConfigTester 2020-01-10 21:36:18 +00:00
Julien Chaumond fd8423321f keep list sorted 2020-01-10 20:36:46 +00:00
Julien Chaumond 0cd81fb99f [isort] declare more third-parties in case no tf install 2020-01-10 20:35:45 +00:00
Julien Chaumond 84c0aa1868 num_parameters helper 2020-01-10 17:40:02 +00:00
Julien Chaumond f599623a99 PreTrainedTokenizerFast: hotfix _convert_encoding 
cc @n1t0
 2020-01-08 15:46:37 -05:00