c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,722 Commits 171 Branches 49 Tags
f8c1071c51dc74fe8a8706585509b0f6d32f3a95
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond cbad305ce6 [docs] The use of do_lower_case in scripts is on its way to deprecation (#3738) 2020-04-10 12:34:04 -04:00
Julien Chaumond b169ac9c2b [examples] Generate argparsers from type hints on dataclasses (#3669) 
* [examples] Generate argparsers from type hints on dataclasses

* [HfArgumentParser] way simpler API

* Restore run_language_modeling.py for easier diff

* [HfArgumentParser] final tweaks from code review
 2020-04-10 12:21:58 -04:00
Julien Chaumond f98d0ef2a2 Big cleanup of glue_convert_examples_to_features (#3688) 
* Big cleanup of `glue_convert_examples_to_features`

* Use batch_encode_plus

* Cleaner wrapping of glue_convert_examples_to_features for TF

@lysandrejik

* Cleanup syntax, thanks to @mfuntowicz

* Raise explicit error in case of user error
 2020-04-10 10:20:18 -04:00
Julien Chaumond bc65afc4df [Exbert] Change style of button 2020-04-09 10:44:42 -04:00
Julien Chaumond a594ee9c84 More doc for model cards (#3698) 
see https://github.com/huggingface/transformers/pull/3679#pullrequestreview-389368270
 2020-04-08 12:12:52 -04:00
Julien Chaumond 83703cd077 Update doc for {Summarization,Translation}Pipeline and other tweaks 2020-04-08 09:45:00 -04:00
Julien Chaumond 11cc1e168b [model_cards] Turn down spurious warnings 
Close #3639 + spurious warning mentioned in #3227

cc @lysandrejik @thomwolf
 2020-04-07 10:20:19 -04:00
Julien ChaumondKevin ClarkLysandre Debut
 		39a34cc375 [model_cards] ELECTRA (w/ examples of usage) 
Co-Authored-By: Kevin Clark <clarkkev@users.noreply.github.com>
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-04-06 11:43:33 -04:00
Julien Chaumond 5d912e7ed4 Tweak typing for #3566 2020-04-04 15:04:03 -04:00
Julien Chaumond 94eb68d742 weigths*weights 2020-04-04 15:03:26 -04:00
Julien Chaumond 3e4b4dd190 [model_cards] Link to ExBERT visualisation 
Hat/tip @bhoov @HendrikStrobelt @sebastianGehrmann

Also cc @srush and @thomwolf
 2020-04-03 20:03:29 -04:00
Julien Chaumond 50e15c825c Tokenizers: Start cleaning examples a little (#3455) 
* Start cleaning examples

* Fixup
 2020-04-01 07:13:40 -04:00
Julien Chaumond cc598b312b [InputExample] Unfreeze for now, cf. #3423 2020-03-30 10:41:49 -04:00
Julien Chaumond 83272a3853 Experiment w/ dataclasses (including Py36) (#3423) 
* [ci] Also run test_examples in py37

(will revert at the end of the experiment)

* InputExample: use immutable dataclass

* [deps] Install dataclasses for Py<3.7

* [skip ci] Revert "[ci] Also run test_examples in py37"

This reverts commit d29afd9959786b77759b0b8fa4e6b4335b952015.
 2020-03-25 11:10:20 -04:00
Julien Chaumond eaabaaf750 [run_language_modeling] Fix: initialize a new model from a config object 2020-03-24 17:56:40 -04:00
Julien Chaumond f8823bad9a Expose missing mappings (see #3415) 2020-03-24 17:46:25 -04:00
Julien Chaumond d0c36a7b72 [ci] Partial revert of 18eec3a984 due to fbc5bf10cf 2020-03-24 12:10:43 -04:00
Julien Chaumond a8e3336a85 [examples] Use AutoModels in more examples 2020-03-23 20:11:14 -04:00
Julien Chaumond ec6766a363 [deps] scikit-learn's transient issue was fixed 2020-03-23 18:38:09 -04:00
Julien Chaumond f7dcf8fcea [BertAbs] Move files around for more consistent naming 2020-03-23 13:58:49 -04:00
Julien Chaumond e25c4f4027 [ALBERT] move things around for more consistent naming 
see #3359

cc @lysandrejik
 2020-03-23 13:58:21 -04:00
Julien Chaumond 18eec3a984 [ci] simpler way to load correct version of isort 
hat/tip @bramvanroy
 2020-03-23 10:03:22 -04:00
Julien Chaumond cf72479bf1 One last reorder of {scheduler,optimizer}.step() 2020-03-20 18:05:50 -04:00
Julien Chaumond ecfd336318 Simpler Error message when loading config/model with .from_pretrained() (#3341) 2020-03-19 23:23:03 +01:00
Julien Chaumond 656e1386a2 Fix #3305: run_ner only possible on ModelForTokenClassification models 2020-03-19 16:41:28 -04:00
Julien Chaumond 0f1bc0d68e [model_cards] Add google thumbnail 2020-03-17 12:02:51 -04:00
Julien Chaumond 68ef0a111f [model_cards] Symlink all Google AI's BERT Miniatures to source model card 2020-03-16 23:37:42 -04:00
Julien Chaumond 8320feec09 [model_cards] CodeBERTa 2020-03-13 18:28:09 -04:00
Julien ChaumondandDarek Kłeczek 72768b6b9c [model_cards] polbert: simplify usage example with pipelines 
Co-Authored-By: Darek Kłeczek <darek.kleczek@gmail.com>
 2020-03-12 10:05:40 -04:00
Julien Chaumond a4c75f1492 [ci] last resort 2020-03-11 19:11:19 -04:00
Julien Chaumond 824e320d96 [ci] Fixup c6cf925 2020-03-11 18:52:10 -04:00
Julien Chaumond c6cf925ff8 [ci] last resort 
while looking for fix to https://twitter.com/julien_c/status/1237864185821708291
 2020-03-11 18:49:19 -04:00
Julien ChaumondandPatrick von Platen e43afb1bb8 [model_cards] DialoGPT: How to use + thumbnail + conversational tag 
cc @dreasysnail

Co-Authored-By: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-03-11 11:36:47 -04:00
Julien Chaumond 5085df995f [model_cards] PolBERT tweaks 2020-03-11 09:29:22 -04:00
Julien ChaumondandThomas Wolf d6de6423ba [doc] --organization tweak 
Co-Authored-By: Thomas Wolf <thomwolf@users.noreply.github.com>
 2020-03-10 16:52:44 -04:00
Julien Chaumond 0e56dc3078 [doc] Document the new --organization flag of CLI 2020-03-10 16:42:01 -04:00
Julien Chaumond 270dfa1c8e [dialogpt] conversion script 
Reference: https://github.com/huggingface/transformers/pull/1778#issuecomment-567675530

cc @patrickvonplaten and @dreasysnail
 2020-03-10 15:09:29 -04:00
Julien Chaumond cbf8f5d32b [model upload] Support for organizations 2020-03-09 17:33:57 -04:00
Julien Chaumond e03129ad44 [model_cards] Small formatting fix 
cc @mrm8488
 2020-03-06 18:07:44 -05:00
Julien Chaumond 08a70fb392 [model_cards] Fixup d6df9a8f 2020-03-06 17:50:38 -05:00
Julien Chaumond 8a2d9bc9ef Add model cards for DeepPavlov models (#3138) 
* add empty model cards for every current DeepPavlov model

* fix: replace cyrillic `с` with `c`

* docs: add model cards for current DeepPavlov BERT models

* docs: add links for arXiv preprints
 2020-03-05 09:34:43 -05:00
Julien Chaumond 3f067f4409 [hf_api] slightly more doc 2020-03-04 23:55:46 -05:00
Julien Chaumond f564f93c84 [hf_api] Get the public list of all the models on huggingface 2020-03-04 23:33:09 -05:00
Julien Chaumond ff9e79ba3a make style 2020-03-04 20:18:07 -05:00
Julien Chaumond c440030e99 [model_cards] Tag AR model languages 2020-03-04 16:33:10 -05:00
Julien Chaumond 76111a3d3a [model_cards] Add card by @lvwerra 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:55:20 -05:00
Julien Chaumond a43c388abb [model_cards] Add card by @djstrong 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:53:02 -05:00
Julien Chaumond 256cbbc4a2 [doc] Fix link to how-to-train Colab 2020-03-04 12:01:45 -05:00
Julien Chaumond f631e01d2c [ci] Re-run integration ground truth from fairseq 
Adopted best practice set by @patrickvonplaten of commenting lines run on fairseq, for easy comparison

also see #3020
 2020-03-03 15:31:40 -05:00
Julien Chaumond a088d75e51 [model_cards] Fix incorrect path 2020-03-03 09:52:32 -05:00
Julien Chaumond eec5ec8071 [BART] to each its own config + make BART compatible w/ Pipelines 
cc @sshleifer
 2020-03-02 18:56:17 -05:00
Julien Chaumond f169957d0c TF GPU CI (#3085) 
* debug env

* Restrict TF GPU memory

* Fixup

* One more test

* rm debug logs

* Fixup
 2020-03-02 15:45:25 -05:00
Julien Chaumond 0e56b37e80 rm bogus file 
cc @patrickvonplaten
 2020-03-02 12:27:12 -05:00
Julien ChaumondFuntowicz MorganLysandre Debut
 		13afb71208 [ci] Ensure that TF does not preempt all GPU memory for itself 
see https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth

Co-Authored-By: Funtowicz Morgan <mfuntowicz@users.noreply.github.com>
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-03-02 11:56:45 -05:00
Julien Chaumond 298bed16a8 make style 2020-03-01 14:08:01 -05:00
Julien Chaumond d6ef587a10 [ci] Fixup e36bd94345 2020-02-28 23:19:17 -05:00
Julien Chaumond e36bd94345 [ci] Run all tests on (self-hosted) GPU (#3020) 
* Create self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* do not run slow tests, for now

* [ci] For comparison with circleci, let's also run CPU-tests

* [ci] reorganize

* clearer filenames

* [ci] Final tweaks before merging

* rm slow tests on circle ci

* Trigger CI

* On GPU this concurrency was way too high
 2020-02-28 21:11:08 -05:00
Julien Chaumond b370cc7e99 [gpu] Fixup fdd61b1992 2020-02-26 21:48:49 +00:00
Julien Chaumond f5516805c2 Fix bart slow test 2020-02-26 20:47:49 +00:00
Julien ChaumondandSam Shleifer 9cda3620b6 Fix (non-slow) tests on GPU (torch) (#3024) 
* Fix tests on GPU (torch)

* Fix bart slow tests

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-02-26 11:59:25 -05:00
Julien Chaumond e693cd1e87 [ci] Run slow tests every day 2020-02-24 19:54:47 -05:00
Julien Chaumond 4fc63151af [ci] Attempt to fix #2844 2020-02-24 19:51:34 -05:00
Julien Chaumond 73028c5df0 [model_cards] EsperBERTo 2020-02-14 15:16:33 -05:00
Julien ChaumondandIlias Chalkidis 4e69104a1f [model_cards] Also use the thumbnail as meta 
Co-Authored-By: Ilias Chalkidis <ihalk@di.uoa.gr>
 2020-02-14 10:27:11 -05:00
Julien ChaumondandYves Peirsman 73d79d42b4 [model_cards] nlptown/bert-base-multilingual-uncased-sentiment 
cc @yvespeirsman

Co-Authored-By: Yves Peirsman <yvespeirsman@users.noreply.github.com>
 2020-02-14 09:51:11 -05:00
Julien Chaumond 7d22fefd37 [pipeline] Alias NerPipeline as TokenClassificationPipeline 2020-02-14 09:18:10 -05:00
Julien Chaumond 925a13ced1 [model_cards] mv README.md 2020-02-13 23:07:29 -05:00
Julien Chaumond 4d36472b96 [run_ner] Don't crash if fine-tuning local model that doesn't end with digit 2020-02-14 03:25:29 +00:00
Julien Chaumond 21da895013 [model_cards] Better image for social sharing 2020-02-11 20:30:08 -05:00
Julien Chaumond 9a70910d47 [model_cards] Tweak @mrm8488's model card 2020-02-11 20:20:39 -05:00
Julien Chaumond 9274734a0d [model_cards] mv to correct location + tweak tag 2020-02-11 20:13:57 -05:00
Julien Chaumond e0b6247cf7 [model_cards] Change formatting slightly as we updated our markdown engine 
cc @tholor @loretoparisi @simonefrancia
 2020-02-11 18:25:21 -05:00
Julien Chaumond 1f5db9a13c [model_cards] Rm extraneous tag 2020-02-10 17:45:13 -05:00
Julien Chaumond 95bac8dabb [model_cards] Add language metadata to existing model cards 
This will enable filtering on language (amongst other tags) on the website

cc @loretoparisi, @stefan-it, @HenrykBorzymowski, @marma
 2020-02-10 17:42:42 -05:00
Julien ChaumondandKevin Canwen Xu b4691a438d [model_cards] BERT-of-Theseus: use the visual as thumbnail 
cc @jetrunner

Co-Authored-By: Kevin Canwen Xu <canwenxu@outlook.com>
 2020-02-10 11:27:08 -05:00
Julien Chaumond fc325e97cd [model_cards] Showcase model tag syntax 2020-02-10 11:27:08 -05:00
Julien Chaumond 63a5399bc4 [model_cards] Specify language meta + thumbnail 
cc @tholor

see #2799
 2020-02-10 11:20:05 -05:00
Julien Chaumond 42f08e596f [examples] rename run_lm_finetuning to run_language_modeling 2020-02-07 09:15:28 -05:00
Julien Chaumond 4f7bdb0958 [examples] Fix broken markdown 2020-02-07 09:15:28 -05:00
Julien Chaumond 7748cbbe7d Oopsie 2020-02-06 15:30:02 -05:00
Julien Chaumond 432c12521e [docs] Add menu w/ links to other pages on hf.co 2020-02-06 15:30:02 -05:00
Julien Chaumond eae8ee0389 [doc] model sharing: mention README.md + tweaks 
cc @lysandrejik @thomwolf
 2020-02-05 14:20:03 -05:00
Julien ChaumondandLysandre Debut ada24def22 [run_lm_finetuning] Tweak fix for non-long tensor, close #2728 
see 1ebfeb7946 and #2728

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-02-05 12:49:18 -05:00
Julien Chaumond 9a50828b5c Pipelines: fix crash when modelcard is None 
cc @mfuntowicz does this seem correct?
 2020-02-03 17:53:39 -05:00
Julien ChaumondandStefan Schweter ddb6f9476b [model_cards] dbmdz models 
Co-Authored-By: Stefan Schweter <stefan-it@users.noreply.github.com>
 2020-01-31 18:39:09 -05:00
Julien ChaumondandHenrykBorzymowski 6636826f04 [model_cards] Multilingual + Dutch SQuAD2.0 
Co-Authored-By: HenrykBorzymowski <henrykborzymowski@users.noreply.github.com>
 2020-01-31 18:39:09 -05:00
Julien ChaumondLoreto ParisiSimone Francia
 		98dadc98e1 [model_cards] UmBERTo 
Co-Authored-By: Loreto Parisi <loretoparisi@gmail.com>
Co-Authored-By: Simone Francia <francia.simone1@gmail.com>
 2020-01-31 18:39:09 -05:00
Julien Chaumond d6fc34b459 [model_cards] add mine 2020-01-31 18:39:09 -05:00
Julien Chaumond 5a6b138b00 [Umberto] model shortcuts (#2661) 
* [Umberto] model shortcuts

cc @loretoparisi @simonefrancia

see #2485

* Ensure that tokenizers will be correctly configured
 2020-01-30 21:05:53 -05:00
Julien Chaumond 7fe294bf07 Hotfix: same handling of non-existent files as for config 2020-01-30 20:05:04 -05:00
Julien Chaumond b85c59f997 config.architectures 2020-01-30 19:26:59 -05:00
Julien Chaumond f9bc3f5771 style tweak 2020-01-30 19:26:59 -05:00
Julien Chaumond 0b13fb822a No need for a model_type here 
cc @lysandrejik
 2020-01-30 19:26:59 -05:00
Julien Chaumond 9fa836a73f fill_mask helper (#2576) 
* fill_mask helper

* [poc] FillMaskPipeline

* Revert "[poc] FillMaskPipeline"

This reverts commit 67eeea55b0f97b46c2b828de0f4ee97d87338335.

* Revert "fill_mask helper"

This reverts commit cacc17b884e14bb6b07989110ffe884ad9e36eaa.

* README: clarify that Pipelines can also do text-classification

cf. question at the AI&ML meetup last week, @mfuntowicz

* Fix test: test feature-extraction pipeline

* Test tweaks

* Slight refactor of existing pipeline (in preparation of new FillMaskPipeline)

* Extraneous doc

* More robust way of doing this

@mfuntowicz as we don't rely on the model name anymore (see AutoConfig)

* Also add RobertaConfig as a quickfix for wrong token_type_ids

* cs

* [BIG] FillMaskPipeline
 2020-01-30 18:15:42 -05:00
Julien Chaumond 6b4c3ee234 [run_lm_finetuning] GPT2 tokenizer doesn't have a pad_token 
ping @lysandrejik
 2020-01-27 20:14:02 -05:00
Julien Chaumond 79815bf666 [serving] Fix typo 2020-01-27 19:58:25 -05:00
Julien Chaumond 5004d5af42 [serving] Update dependencies 2020-01-27 19:58:00 -05:00
Julien Chaumond ea56d305be make style 2020-01-27 12:13:32 -05:00
Julien Chaumond 11b13e94a3 Add type to help my IDE out 2020-01-24 14:00:57 -05:00
Julien Chaumond 90b7df444f Upload CLI: on win32, use slashes, not os.sep 2020-01-22 22:41:21 -05:00