c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,570 Commits 171 Branches 49 Tags
85b324bee56c32142cc131149d7a92281964641f
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond 18eec3a984 [ci] simpler way to load correct version of isort 
hat/tip @bramvanroy
 2020-03-23 10:03:22 -04:00
Julien Chaumond cf72479bf1 One last reorder of {scheduler,optimizer}.step() 2020-03-20 18:05:50 -04:00
Julien Chaumond ecfd336318 Simpler Error message when loading config/model with .from_pretrained() (#3341) 2020-03-19 23:23:03 +01:00
Julien Chaumond 656e1386a2 Fix #3305: run_ner only possible on ModelForTokenClassification models 2020-03-19 16:41:28 -04:00
Julien Chaumond 0f1bc0d68e [model_cards] Add google thumbnail 2020-03-17 12:02:51 -04:00
Julien Chaumond 68ef0a111f [model_cards] Symlink all Google AI's BERT Miniatures to source model card 2020-03-16 23:37:42 -04:00
Julien Chaumond 8320feec09 [model_cards] CodeBERTa 2020-03-13 18:28:09 -04:00
Julien ChaumondandDarek Kłeczek 72768b6b9c [model_cards] polbert: simplify usage example with pipelines 
Co-Authored-By: Darek Kłeczek <darek.kleczek@gmail.com>
 2020-03-12 10:05:40 -04:00
Julien Chaumond a4c75f1492 [ci] last resort 2020-03-11 19:11:19 -04:00
Julien Chaumond 824e320d96 [ci] Fixup c6cf925 2020-03-11 18:52:10 -04:00
Julien Chaumond c6cf925ff8 [ci] last resort 
while looking for fix to https://twitter.com/julien_c/status/1237864185821708291
 2020-03-11 18:49:19 -04:00
Julien ChaumondandPatrick von Platen e43afb1bb8 [model_cards] DialoGPT: How to use + thumbnail + conversational tag 
cc @dreasysnail

Co-Authored-By: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-03-11 11:36:47 -04:00
Julien Chaumond 5085df995f [model_cards] PolBERT tweaks 2020-03-11 09:29:22 -04:00
Julien ChaumondandThomas Wolf d6de6423ba [doc] --organization tweak 
Co-Authored-By: Thomas Wolf <thomwolf@users.noreply.github.com>
 2020-03-10 16:52:44 -04:00
Julien Chaumond 0e56dc3078 [doc] Document the new --organization flag of CLI 2020-03-10 16:42:01 -04:00
Julien Chaumond 270dfa1c8e [dialogpt] conversion script 
Reference: https://github.com/huggingface/transformers/pull/1778#issuecomment-567675530

cc @patrickvonplaten and @dreasysnail
 2020-03-10 15:09:29 -04:00
Julien Chaumond cbf8f5d32b [model upload] Support for organizations 2020-03-09 17:33:57 -04:00
Julien Chaumond e03129ad44 [model_cards] Small formatting fix 
cc @mrm8488
 2020-03-06 18:07:44 -05:00
Julien Chaumond 08a70fb392 [model_cards] Fixup d6df9a8f 2020-03-06 17:50:38 -05:00
Julien Chaumond 8a2d9bc9ef Add model cards for DeepPavlov models (#3138) 
* add empty model cards for every current DeepPavlov model

* fix: replace cyrillic `с` with `c`

* docs: add model cards for current DeepPavlov BERT models

* docs: add links for arXiv preprints
 2020-03-05 09:34:43 -05:00
Julien Chaumond 3f067f4409 [hf_api] slightly more doc 2020-03-04 23:55:46 -05:00
Julien Chaumond f564f93c84 [hf_api] Get the public list of all the models on huggingface 2020-03-04 23:33:09 -05:00
Julien Chaumond ff9e79ba3a make style 2020-03-04 20:18:07 -05:00
Julien Chaumond c440030e99 [model_cards] Tag AR model languages 2020-03-04 16:33:10 -05:00
Julien Chaumond 76111a3d3a [model_cards] Add card by @lvwerra 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:55:20 -05:00
Julien Chaumond a43c388abb [model_cards] Add card by @djstrong 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:53:02 -05:00
Julien Chaumond 256cbbc4a2 [doc] Fix link to how-to-train Colab 2020-03-04 12:01:45 -05:00
Julien Chaumond f631e01d2c [ci] Re-run integration ground truth from fairseq 
Adopted best practice set by @patrickvonplaten of commenting lines run on fairseq, for easy comparison

also see #3020
 2020-03-03 15:31:40 -05:00
Julien Chaumond a088d75e51 [model_cards] Fix incorrect path 2020-03-03 09:52:32 -05:00
Julien Chaumond eec5ec8071 [BART] to each its own config + make BART compatible w/ Pipelines 
cc @sshleifer
 2020-03-02 18:56:17 -05:00
Julien Chaumond f169957d0c TF GPU CI (#3085) 
* debug env

* Restrict TF GPU memory

* Fixup

* One more test

* rm debug logs

* Fixup
 2020-03-02 15:45:25 -05:00
Julien Chaumond 0e56b37e80 rm bogus file 
cc @patrickvonplaten
 2020-03-02 12:27:12 -05:00
Julien ChaumondFuntowicz MorganLysandre Debut
 		13afb71208 [ci] Ensure that TF does not preempt all GPU memory for itself 
see https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth

Co-Authored-By: Funtowicz Morgan <mfuntowicz@users.noreply.github.com>
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-03-02 11:56:45 -05:00
Julien Chaumond 298bed16a8 make style 2020-03-01 14:08:01 -05:00
Julien Chaumond d6ef587a10 [ci] Fixup e36bd94345 2020-02-28 23:19:17 -05:00
Julien Chaumond e36bd94345 [ci] Run all tests on (self-hosted) GPU (#3020) 
* Create self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* do not run slow tests, for now

* [ci] For comparison with circleci, let's also run CPU-tests

* [ci] reorganize

* clearer filenames

* [ci] Final tweaks before merging

* rm slow tests on circle ci

* Trigger CI

* On GPU this concurrency was way too high
 2020-02-28 21:11:08 -05:00
Julien Chaumond b370cc7e99 [gpu] Fixup fdd61b1992 2020-02-26 21:48:49 +00:00
Julien Chaumond f5516805c2 Fix bart slow test 2020-02-26 20:47:49 +00:00
Julien ChaumondandSam Shleifer 9cda3620b6 Fix (non-slow) tests on GPU (torch) (#3024) 
* Fix tests on GPU (torch)

* Fix bart slow tests

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-02-26 11:59:25 -05:00
Julien Chaumond e693cd1e87 [ci] Run slow tests every day 2020-02-24 19:54:47 -05:00
Julien Chaumond 4fc63151af [ci] Attempt to fix #2844 2020-02-24 19:51:34 -05:00
Julien Chaumond 73028c5df0 [model_cards] EsperBERTo 2020-02-14 15:16:33 -05:00
Julien ChaumondandIlias Chalkidis 4e69104a1f [model_cards] Also use the thumbnail as meta 
Co-Authored-By: Ilias Chalkidis <ihalk@di.uoa.gr>
 2020-02-14 10:27:11 -05:00
Julien ChaumondandYves Peirsman 73d79d42b4 [model_cards] nlptown/bert-base-multilingual-uncased-sentiment 
cc @yvespeirsman

Co-Authored-By: Yves Peirsman <yvespeirsman@users.noreply.github.com>
 2020-02-14 09:51:11 -05:00
Julien Chaumond 7d22fefd37 [pipeline] Alias NerPipeline as TokenClassificationPipeline 2020-02-14 09:18:10 -05:00
Julien Chaumond 925a13ced1 [model_cards] mv README.md 2020-02-13 23:07:29 -05:00
Julien Chaumond 4d36472b96 [run_ner] Don't crash if fine-tuning local model that doesn't end with digit 2020-02-14 03:25:29 +00:00
Julien Chaumond 21da895013 [model_cards] Better image for social sharing 2020-02-11 20:30:08 -05:00
Julien Chaumond 9a70910d47 [model_cards] Tweak @mrm8488's model card 2020-02-11 20:20:39 -05:00
Julien Chaumond 9274734a0d [model_cards] mv to correct location + tweak tag 2020-02-11 20:13:57 -05:00
Julien Chaumond e0b6247cf7 [model_cards] Change formatting slightly as we updated our markdown engine 
cc @tholor @loretoparisi @simonefrancia
 2020-02-11 18:25:21 -05:00
Julien Chaumond 1f5db9a13c [model_cards] Rm extraneous tag 2020-02-10 17:45:13 -05:00
Julien Chaumond 95bac8dabb [model_cards] Add language metadata to existing model cards 
This will enable filtering on language (amongst other tags) on the website

cc @loretoparisi, @stefan-it, @HenrykBorzymowski, @marma
 2020-02-10 17:42:42 -05:00
Julien ChaumondandKevin Canwen Xu b4691a438d [model_cards] BERT-of-Theseus: use the visual as thumbnail 
cc @jetrunner

Co-Authored-By: Kevin Canwen Xu <canwenxu@outlook.com>
 2020-02-10 11:27:08 -05:00
Julien Chaumond fc325e97cd [model_cards] Showcase model tag syntax 2020-02-10 11:27:08 -05:00
Julien Chaumond 63a5399bc4 [model_cards] Specify language meta + thumbnail 
cc @tholor

see #2799
 2020-02-10 11:20:05 -05:00
Julien Chaumond 42f08e596f [examples] rename run_lm_finetuning to run_language_modeling 2020-02-07 09:15:28 -05:00
Julien Chaumond 4f7bdb0958 [examples] Fix broken markdown 2020-02-07 09:15:28 -05:00
Julien Chaumond 7748cbbe7d Oopsie 2020-02-06 15:30:02 -05:00
Julien Chaumond 432c12521e [docs] Add menu w/ links to other pages on hf.co 2020-02-06 15:30:02 -05:00
Julien Chaumond eae8ee0389 [doc] model sharing: mention README.md + tweaks 
cc @lysandrejik @thomwolf
 2020-02-05 14:20:03 -05:00
Julien ChaumondandLysandre Debut ada24def22 [run_lm_finetuning] Tweak fix for non-long tensor, close #2728 
see 1ebfeb7946 and #2728

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-02-05 12:49:18 -05:00
Julien Chaumond 9a50828b5c Pipelines: fix crash when modelcard is None 
cc @mfuntowicz does this seem correct?
 2020-02-03 17:53:39 -05:00
Julien ChaumondandStefan Schweter ddb6f9476b [model_cards] dbmdz models 
Co-Authored-By: Stefan Schweter <stefan-it@users.noreply.github.com>
 2020-01-31 18:39:09 -05:00
Julien ChaumondandHenrykBorzymowski 6636826f04 [model_cards] Multilingual + Dutch SQuAD2.0 
Co-Authored-By: HenrykBorzymowski <henrykborzymowski@users.noreply.github.com>
 2020-01-31 18:39:09 -05:00
Julien ChaumondLoreto ParisiSimone Francia
 		98dadc98e1 [model_cards] UmBERTo 
Co-Authored-By: Loreto Parisi <loretoparisi@gmail.com>
Co-Authored-By: Simone Francia <francia.simone1@gmail.com>
 2020-01-31 18:39:09 -05:00
Julien Chaumond d6fc34b459 [model_cards] add mine 2020-01-31 18:39:09 -05:00
Julien Chaumond 5a6b138b00 [Umberto] model shortcuts (#2661) 
* [Umberto] model shortcuts

cc @loretoparisi @simonefrancia

see #2485

* Ensure that tokenizers will be correctly configured
 2020-01-30 21:05:53 -05:00
Julien Chaumond 7fe294bf07 Hotfix: same handling of non-existent files as for config 2020-01-30 20:05:04 -05:00
Julien Chaumond b85c59f997 config.architectures 2020-01-30 19:26:59 -05:00
Julien Chaumond f9bc3f5771 style tweak 2020-01-30 19:26:59 -05:00
Julien Chaumond 0b13fb822a No need for a model_type here 
cc @lysandrejik
 2020-01-30 19:26:59 -05:00
Julien Chaumond 9fa836a73f fill_mask helper (#2576) 
* fill_mask helper

* [poc] FillMaskPipeline

* Revert "[poc] FillMaskPipeline"

This reverts commit 67eeea55b0f97b46c2b828de0f4ee97d87338335.

* Revert "fill_mask helper"

This reverts commit cacc17b884e14bb6b07989110ffe884ad9e36eaa.

* README: clarify that Pipelines can also do text-classification

cf. question at the AI&ML meetup last week, @mfuntowicz

* Fix test: test feature-extraction pipeline

* Test tweaks

* Slight refactor of existing pipeline (in preparation of new FillMaskPipeline)

* Extraneous doc

* More robust way of doing this

@mfuntowicz as we don't rely on the model name anymore (see AutoConfig)

* Also add RobertaConfig as a quickfix for wrong token_type_ids

* cs

* [BIG] FillMaskPipeline
 2020-01-30 18:15:42 -05:00
Julien Chaumond 6b4c3ee234 [run_lm_finetuning] GPT2 tokenizer doesn't have a pad_token 
ping @lysandrejik
 2020-01-27 20:14:02 -05:00
Julien Chaumond 79815bf666 [serving] Fix typo 2020-01-27 19:58:25 -05:00
Julien Chaumond 5004d5af42 [serving] Update dependencies 2020-01-27 19:58:00 -05:00
Julien Chaumond ea56d305be make style 2020-01-27 12:13:32 -05:00
Julien Chaumond 11b13e94a3 Add type to help my IDE out 2020-01-24 14:00:57 -05:00
Julien Chaumond 90b7df444f Upload CLI: on win32, use slashes, not os.sep 2020-01-22 22:41:21 -05:00
Julien Chaumond 119dc50e2a Doc tweak on model sharing 2020-01-22 22:40:38 -05:00
Julien Chaumond 34a3c25a30 Fix for XLMRobertaConfig inherits from RobertaConfig 
hat/tip @stefan-it
 2020-01-22 17:50:24 -05:00
Julien Chaumond 1a8e87be4e Line-by-line text dataset (including padding) 2020-01-21 16:57:38 -05:00
Julien Chaumond b94cf7faac change order 2020-01-21 16:57:38 -05:00
Julien Chaumond 2eaa8b6e56 Easier to not support this, as it could be confusing 
cc @lysandrejik
 2020-01-21 16:57:38 -05:00
Julien Chaumond 801aaa5508 make style 2020-01-21 16:57:38 -05:00
Julien Chaumond 56d4ba8ddb [run_lm_finetuning] Train from scratch 2020-01-21 16:57:38 -05:00
Julien Chaumond 23a2cea8cb Tokenizer.from_pretrained: fetch all possible files remotely 2020-01-16 16:47:19 -05:00
Julien Chaumond 99f9243de5 same here, try to not serialize too much if unneeded 2020-01-16 16:47:19 -05:00
Julien Chaumond 9d8fd2d40e tokenizer.save_pretrained: only save file if non-empty 2020-01-16 16:47:19 -05:00
Julien Chaumond d9fa1bad72 Fix failing torchscript test for xlnet 
model.parameters() order is apparently not stable (only for xlnet, for some reason)
 2020-01-15 20:22:21 -05:00
Julien Chaumond a98b2ca8c0 Style + fixup BertJapaneseTokenizer 2020-01-15 19:05:51 -05:00
Julien Chaumond 83a41d39b3 💄 super 2020-01-15 18:33:50 -05:00
Julien Chaumond cd51893d37 Merge branch 'Rexhaif-patch-1' 2020-01-15 18:25:15 -05:00
Julien Chaumond 248aeaa842 Merge branch 'patch-1' of https://github.com/Rexhaif/transformers into Rexhaif-patch-1 2020-01-15 18:22:01 -05:00
Julien Chaumond eb59e9f705 Graduate sst-2 to a canonical one 2020-01-15 16:28:50 +00:00
Julien Chaumond e184ad13cf Close #2392 2020-01-15 15:43:44 +00:00
Julien Chaumond 715fa638a7 Merge branch 'master' into from_scratch_training 2020-01-14 18:58:21 +00:00
Julien Chaumond 764f836d52 Update test_tokenization_auto.py 2020-01-13 22:50:34 -05:00
Julien Chaumond d5831acb07 Update test_tokenization_auto.py 2020-01-13 22:47:33 -05:00
Julien Chaumond ed6cd597cc Update test_tokenization_auto.py 2020-01-13 22:46:35 -05:00