c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,871 Commits 171 Branches 49 Tags
ebf80e2e70ebe0054b28eb728c08591e5b488175
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond fd2174664c [Trainer] W&B: Enable model watch 
See https://github.com/huggingface/transformers/pull/3916
 2020-05-05 10:59:23 -04:00
Julien Chaumond d713cfc5eb GePpeTto 🇮🇹: Fixpath to model card 2020-05-01 11:48:58 -04:00
Julien Chaumond 27d55125e6 Configs: saner num_labels in configs. (#3967) 2020-05-01 11:28:55 -04:00
Julien Chaumond b8686174be Merge pull request #3934 from huggingface/examples_args_from_files 
[qol] example scripts: parse args from .args file or JSON
 2020-04-30 22:40:13 -04:00
Julien Chaumond f39217a5ec [tests] Light cleanup of tempfile in tests/ 2020-04-30 22:30:15 -04:00
Julien Chaumond f54dc3f4d5 [ci] Load pretrained models into the default (long-lived) cache 
There's an inconsistency right now where:
- we load some models into CACHE_DIR
- and some models in the default cache
- and often, in both for the same models

When running the RUN_SLOW tests, this takes a lot of disk space, time, and bandwidth.

I'd rather always use the default cache
 2020-04-30 22:30:15 -04:00
Julien Chaumond ab90353f1a [cli] {login, upload, s3} display more helpful error messages 2020-04-30 12:51:06 -04:00
Julien Chaumond 452dd0e4d9 [ci] Align test_hf_api.py with API change 2020-04-30 12:06:01 -04:00
Julien Chaumond 6faca88ee0 Align MarianMT with #4030 
cc @sshleifer
 2020-04-28 20:35:20 -04:00
Julien Chaumond 211e130811 [github] Issue templates: populate some labels 
cc @bramvanroy @stefan-it
 2020-04-28 20:34:34 -04:00
Julien Chaumond 455c639093 CDN urls (#4030) 
* [file_utils] use_cdn + documentation

* Move to cdn. urls for weights

* [urls] Hotfix for bert-base-japanese
 2020-04-28 20:27:14 -04:00
Julien Chaumond 97a375484c rm boto3 dependency 2020-04-27 11:17:14 -04:00
Julien Chaumond c811526004 [examples] For convenience, also save the tokenizer 
Close #3921
 2020-04-24 09:52:42 -04:00
Julien Chaumond c53cc018de [Trainer] Fix _rotate_checkpoints 
Close #3920
 2020-04-23 23:59:43 +00:00
Julien Chaumond cbbb3c43c5 [hubconf] Modify pythonpath to get canonical imports to work 
See https://github.com/huggingface/transformers/pull/3881/files#r412292660

Should we remove SRC_DIR from sys.path right after the imports, @aaugustin?
 2020-04-23 16:27:43 -04:00
Julien Chaumond 6af5a54c28 [Trainer] reuse constant 2020-04-23 11:02:05 -04:00
Julien Chaumond 7c2a32ff88 [housekeeping] super() 2020-04-23 10:43:22 -04:00
Julien Chaumond a946b6b51b [housekeeping] Upgrade # type Python 2 syntax 
cc @sshleifer
 2020-04-23 10:39:24 -04:00
Julien Chaumond 1dc9b3c784 Fixes #3877 2020-04-22 01:15:10 +00:00
Julien Chaumond dd9d483d03 Trainer (#3800) 
* doc

* [tests] Add sample files for a regression task

* [HUGE] Trainer

* Feedback from @sshleifer

* Feedback from @thomwolf + logging tweak

* [file_utils] when downloading concurrently, get_from_cache will use the cached file for subsequent processes

* [glue] Use default max_seq_length of 128 like before

* [glue] move DataTrainingArguments around

* [ner] Change interface of InputExample, and align run_{tf,pl}

* Re-align the pl scripts a little bit

* ner

* [ner] Add integration test

* Fix language_modeling with API tweak

* [ci] Tweak loss target

* Don't break console output

* amp.initialize: model must be on right device before

* [multiple-choice] update for Trainer

* Re-align to 827d6d6ef0
 2020-04-21 20:11:56 -04:00
Julien Chaumond eb5601b0a5 [ci] Pin torch version while we update 2020-04-21 15:46:18 -04:00
Julien Chaumond d32585a304 Fix Torch.hub + Integration test 2020-04-21 14:13:30 -04:00
Julien Chaumond 60a42ef1c0 [model_cards] Fix CamemBERT table markdown 
see https://github.com/huggingface/transformers/pull/3836
 2020-04-17 20:21:15 -04:00
Julien Chaumond 88aecee6a2 [ci] GitHub-hosted runner has no space left on device 2020-04-17 20:16:00 -04:00
Julien Chaumond cbad305ce6 [docs] The use of do_lower_case in scripts is on its way to deprecation (#3738) 2020-04-10 12:34:04 -04:00
Julien Chaumond b169ac9c2b [examples] Generate argparsers from type hints on dataclasses (#3669) 
* [examples] Generate argparsers from type hints on dataclasses

* [HfArgumentParser] way simpler API

* Restore run_language_modeling.py for easier diff

* [HfArgumentParser] final tweaks from code review
 2020-04-10 12:21:58 -04:00
Julien Chaumond f98d0ef2a2 Big cleanup of glue_convert_examples_to_features (#3688) 
* Big cleanup of `glue_convert_examples_to_features`

* Use batch_encode_plus

* Cleaner wrapping of glue_convert_examples_to_features for TF

@lysandrejik

* Cleanup syntax, thanks to @mfuntowicz

* Raise explicit error in case of user error
 2020-04-10 10:20:18 -04:00
Julien Chaumond bc65afc4df [Exbert] Change style of button 2020-04-09 10:44:42 -04:00
Julien Chaumond a594ee9c84 More doc for model cards (#3698) 
see https://github.com/huggingface/transformers/pull/3679#pullrequestreview-389368270
 2020-04-08 12:12:52 -04:00
Julien Chaumond 83703cd077 Update doc for {Summarization,Translation}Pipeline and other tweaks 2020-04-08 09:45:00 -04:00
Julien Chaumond 11cc1e168b [model_cards] Turn down spurious warnings 
Close #3639 + spurious warning mentioned in #3227

cc @lysandrejik @thomwolf
 2020-04-07 10:20:19 -04:00
Julien ChaumondKevin ClarkLysandre Debut
 		39a34cc375 [model_cards] ELECTRA (w/ examples of usage) 
Co-Authored-By: Kevin Clark <clarkkev@users.noreply.github.com>
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-04-06 11:43:33 -04:00
Julien Chaumond 5d912e7ed4 Tweak typing for #3566 2020-04-04 15:04:03 -04:00
Julien Chaumond 94eb68d742 weigths*weights 2020-04-04 15:03:26 -04:00
Julien Chaumond 3e4b4dd190 [model_cards] Link to ExBERT visualisation 
Hat/tip @bhoov @HendrikStrobelt @sebastianGehrmann

Also cc @srush and @thomwolf
 2020-04-03 20:03:29 -04:00
Julien Chaumond 50e15c825c Tokenizers: Start cleaning examples a little (#3455) 
* Start cleaning examples

* Fixup
 2020-04-01 07:13:40 -04:00
Julien Chaumond cc598b312b [InputExample] Unfreeze for now, cf. #3423 2020-03-30 10:41:49 -04:00
Julien Chaumond 83272a3853 Experiment w/ dataclasses (including Py36) (#3423) 
* [ci] Also run test_examples in py37

(will revert at the end of the experiment)

* InputExample: use immutable dataclass

* [deps] Install dataclasses for Py<3.7

* [skip ci] Revert "[ci] Also run test_examples in py37"

This reverts commit d29afd9959786b77759b0b8fa4e6b4335b952015.
 2020-03-25 11:10:20 -04:00
Julien Chaumond eaabaaf750 [run_language_modeling] Fix: initialize a new model from a config object 2020-03-24 17:56:40 -04:00
Julien Chaumond f8823bad9a Expose missing mappings (see #3415) 2020-03-24 17:46:25 -04:00
Julien Chaumond d0c36a7b72 [ci] Partial revert of 18eec3a984 due to fbc5bf10cf 2020-03-24 12:10:43 -04:00
Julien Chaumond a8e3336a85 [examples] Use AutoModels in more examples 2020-03-23 20:11:14 -04:00
Julien Chaumond ec6766a363 [deps] scikit-learn's transient issue was fixed 2020-03-23 18:38:09 -04:00
Julien Chaumond f7dcf8fcea [BertAbs] Move files around for more consistent naming 2020-03-23 13:58:49 -04:00
Julien Chaumond e25c4f4027 [ALBERT] move things around for more consistent naming 
see #3359

cc @lysandrejik
 2020-03-23 13:58:21 -04:00
Julien Chaumond 18eec3a984 [ci] simpler way to load correct version of isort 
hat/tip @bramvanroy
 2020-03-23 10:03:22 -04:00
Julien Chaumond cf72479bf1 One last reorder of {scheduler,optimizer}.step() 2020-03-20 18:05:50 -04:00
Julien Chaumond ecfd336318 Simpler Error message when loading config/model with .from_pretrained() (#3341) 2020-03-19 23:23:03 +01:00
Julien Chaumond 656e1386a2 Fix #3305: run_ner only possible on ModelForTokenClassification models 2020-03-19 16:41:28 -04:00
Julien Chaumond 0f1bc0d68e [model_cards] Add google thumbnail 2020-03-17 12:02:51 -04:00
Julien Chaumond 68ef0a111f [model_cards] Symlink all Google AI's BERT Miniatures to source model card 2020-03-16 23:37:42 -04:00
Julien Chaumond 8320feec09 [model_cards] CodeBERTa 2020-03-13 18:28:09 -04:00
Julien ChaumondandDarek Kłeczek 72768b6b9c [model_cards] polbert: simplify usage example with pipelines 
Co-Authored-By: Darek Kłeczek <darek.kleczek@gmail.com>
 2020-03-12 10:05:40 -04:00
Julien Chaumond a4c75f1492 [ci] last resort 2020-03-11 19:11:19 -04:00
Julien Chaumond 824e320d96 [ci] Fixup c6cf925 2020-03-11 18:52:10 -04:00
Julien Chaumond c6cf925ff8 [ci] last resort 
while looking for fix to https://twitter.com/julien_c/status/1237864185821708291
 2020-03-11 18:49:19 -04:00
Julien ChaumondandPatrick von Platen e43afb1bb8 [model_cards] DialoGPT: How to use + thumbnail + conversational tag 
cc @dreasysnail

Co-Authored-By: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-03-11 11:36:47 -04:00
Julien Chaumond 5085df995f [model_cards] PolBERT tweaks 2020-03-11 09:29:22 -04:00
Julien ChaumondandThomas Wolf d6de6423ba [doc] --organization tweak 
Co-Authored-By: Thomas Wolf <thomwolf@users.noreply.github.com>
 2020-03-10 16:52:44 -04:00
Julien Chaumond 0e56dc3078 [doc] Document the new --organization flag of CLI 2020-03-10 16:42:01 -04:00
Julien Chaumond 270dfa1c8e [dialogpt] conversion script 
Reference: https://github.com/huggingface/transformers/pull/1778#issuecomment-567675530

cc @patrickvonplaten and @dreasysnail
 2020-03-10 15:09:29 -04:00
Julien Chaumond cbf8f5d32b [model upload] Support for organizations 2020-03-09 17:33:57 -04:00
Julien Chaumond e03129ad44 [model_cards] Small formatting fix 
cc @mrm8488
 2020-03-06 18:07:44 -05:00
Julien Chaumond 08a70fb392 [model_cards] Fixup d6df9a8f 2020-03-06 17:50:38 -05:00
Julien Chaumond 8a2d9bc9ef Add model cards for DeepPavlov models (#3138) 
* add empty model cards for every current DeepPavlov model

* fix: replace cyrillic `с` with `c`

* docs: add model cards for current DeepPavlov BERT models

* docs: add links for arXiv preprints
 2020-03-05 09:34:43 -05:00
Julien Chaumond 3f067f4409 [hf_api] slightly more doc 2020-03-04 23:55:46 -05:00
Julien Chaumond f564f93c84 [hf_api] Get the public list of all the models on huggingface 2020-03-04 23:33:09 -05:00
Julien Chaumond ff9e79ba3a make style 2020-03-04 20:18:07 -05:00
Julien Chaumond c440030e99 [model_cards] Tag AR model languages 2020-03-04 16:33:10 -05:00
Julien Chaumond 76111a3d3a [model_cards] Add card by @lvwerra 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:55:20 -05:00
Julien Chaumond a43c388abb [model_cards] Add card by @djstrong 
(the current way to submit a model card to have it displayed on the website is to open a PR on the `transformers` repo itself)

Thanks for sharing!
 2020-03-04 12:53:02 -05:00
Julien Chaumond 256cbbc4a2 [doc] Fix link to how-to-train Colab 2020-03-04 12:01:45 -05:00
Julien Chaumond f631e01d2c [ci] Re-run integration ground truth from fairseq 
Adopted best practice set by @patrickvonplaten of commenting lines run on fairseq, for easy comparison

also see #3020
 2020-03-03 15:31:40 -05:00
Julien Chaumond a088d75e51 [model_cards] Fix incorrect path 2020-03-03 09:52:32 -05:00
Julien Chaumond eec5ec8071 [BART] to each its own config + make BART compatible w/ Pipelines 
cc @sshleifer
 2020-03-02 18:56:17 -05:00
Julien Chaumond f169957d0c TF GPU CI (#3085) 
* debug env

* Restrict TF GPU memory

* Fixup

* One more test

* rm debug logs

* Fixup
 2020-03-02 15:45:25 -05:00
Julien Chaumond 0e56b37e80 rm bogus file 
cc @patrickvonplaten
 2020-03-02 12:27:12 -05:00
Julien ChaumondFuntowicz MorganLysandre Debut
 		13afb71208 [ci] Ensure that TF does not preempt all GPU memory for itself 
see https://www.tensorflow.org/guide/gpu#limiting_gpu_memory_growth

Co-Authored-By: Funtowicz Morgan <mfuntowicz@users.noreply.github.com>
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2020-03-02 11:56:45 -05:00
Julien Chaumond 298bed16a8 make style 2020-03-01 14:08:01 -05:00
Julien Chaumond d6ef587a10 [ci] Fixup e36bd94345 2020-02-28 23:19:17 -05:00
Julien Chaumond e36bd94345 [ci] Run all tests on (self-hosted) GPU (#3020) 
* Create self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* do not run slow tests, for now

* [ci] For comparison with circleci, let's also run CPU-tests

* [ci] reorganize

* clearer filenames

* [ci] Final tweaks before merging

* rm slow tests on circle ci

* Trigger CI

* On GPU this concurrency was way too high
 2020-02-28 21:11:08 -05:00
Julien Chaumond b370cc7e99 [gpu] Fixup fdd61b1992 2020-02-26 21:48:49 +00:00
Julien Chaumond f5516805c2 Fix bart slow test 2020-02-26 20:47:49 +00:00
Julien ChaumondandSam Shleifer 9cda3620b6 Fix (non-slow) tests on GPU (torch) (#3024) 
* Fix tests on GPU (torch)

* Fix bart slow tests

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-02-26 11:59:25 -05:00
Julien Chaumond e693cd1e87 [ci] Run slow tests every day 2020-02-24 19:54:47 -05:00
Julien Chaumond 4fc63151af [ci] Attempt to fix #2844 2020-02-24 19:51:34 -05:00
Julien Chaumond 73028c5df0 [model_cards] EsperBERTo 2020-02-14 15:16:33 -05:00
Julien ChaumondandIlias Chalkidis 4e69104a1f [model_cards] Also use the thumbnail as meta 
Co-Authored-By: Ilias Chalkidis <ihalk@di.uoa.gr>
 2020-02-14 10:27:11 -05:00
Julien ChaumondandYves Peirsman 73d79d42b4 [model_cards] nlptown/bert-base-multilingual-uncased-sentiment 
cc @yvespeirsman

Co-Authored-By: Yves Peirsman <yvespeirsman@users.noreply.github.com>
 2020-02-14 09:51:11 -05:00
Julien Chaumond 7d22fefd37 [pipeline] Alias NerPipeline as TokenClassificationPipeline 2020-02-14 09:18:10 -05:00
Julien Chaumond 925a13ced1 [model_cards] mv README.md 2020-02-13 23:07:29 -05:00
Julien Chaumond 4d36472b96 [run_ner] Don't crash if fine-tuning local model that doesn't end with digit 2020-02-14 03:25:29 +00:00
Julien Chaumond 21da895013 [model_cards] Better image for social sharing 2020-02-11 20:30:08 -05:00
Julien Chaumond 9a70910d47 [model_cards] Tweak @mrm8488's model card 2020-02-11 20:20:39 -05:00
Julien Chaumond 9274734a0d [model_cards] mv to correct location + tweak tag 2020-02-11 20:13:57 -05:00
Julien Chaumond e0b6247cf7 [model_cards] Change formatting slightly as we updated our markdown engine 
cc @tholor @loretoparisi @simonefrancia
 2020-02-11 18:25:21 -05:00
Julien Chaumond 1f5db9a13c [model_cards] Rm extraneous tag 2020-02-10 17:45:13 -05:00
Julien Chaumond 95bac8dabb [model_cards] Add language metadata to existing model cards 
This will enable filtering on language (amongst other tags) on the website

cc @loretoparisi, @stefan-it, @HenrykBorzymowski, @marma
 2020-02-10 17:42:42 -05:00
Julien ChaumondandKevin Canwen Xu b4691a438d [model_cards] BERT-of-Theseus: use the visual as thumbnail 
cc @jetrunner

Co-Authored-By: Kevin Canwen Xu <canwenxu@outlook.com>
 2020-02-10 11:27:08 -05:00
Julien Chaumond fc325e97cd [model_cards] Showcase model tag syntax 2020-02-10 11:27:08 -05:00