c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,839 Commits 171 Branches 49 Tags
fd1f2115cf232997cd04877e0fe7b5550ccd9ea5
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond 8dda9167de [model_cards] harmonization 2020-11-11 12:42:50 +01:00
Julien Chaumond ad2303a401 Fix style 2020-11-10 14:28:30 +01:00
Julien Chaumond 55e8d0cea2 Update links from s3 to huggingface.co 2020-11-10 14:03:29 +01:00
Julien ChaumondSylvain GuggerSam Shleifer
 		70f622fab4 Model versioning (#8324) 
* fix typo

* rm use_cdn & references, and implement new hf_bucket_url

* I'm pretty sure we don't need to `read` this file

* same here

* [BIG] file_utils.networking: do not gobble up errors anymore

* Fix CI 😇

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Tiny doc tweak

* Add doc + pass kwarg everywhere

* Add more tests and explain

cc @sshleifer let me know if better

Co-Authored-By: Sam Shleifer <sshleifer@gmail.com>

* Also implement revision in pipelines

In the case where we're passing a task name or a string model identifier

* Fix CI 😇

* Fix CI

* [hf_api] new methods + command line implem

* make style

* Final endpoints post-migration

* Fix post-migration

* Py3.6 compat

cc @stefan-it

Thank you @stas00

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-11-10 07:11:02 -05:00
Julien Chaumond 55bc0c599a [model_cards] Switch to a more explicit domain for the media bucket 2020-10-27 18:08:05 +01:00
Julien Chaumond 7087d9b1c0 [model_cards] bert-base-danish Fixup 
#8030
 2020-10-26 09:38:21 +01:00
Julien Chaumond efc4a21ffa Fixup #8025 
Close #8030
 2020-10-26 09:32:07 +01:00
Julien Chaumond ff65beafa3 FillMaskPipeline: support passing top_k on __call__ (#7971) 
* FillMaskPipeline: support passing top_k on __call__

Also move from topk to top_k

* migrate to new param name in tests

* Review from @sgugger
 2020-10-22 12:54:25 -04:00
Julien Chaumond 3479787edc Disable inference API for t5-11b (#7978) 2020-10-22 09:08:37 -04:00
Julien Chaumond a7db81c33f [model_card] t5-11b move disclaimer to top of page 
cc @Narsil @patrickvonplaten
 2020-10-22 14:35:31 +02:00
Julien Chaumond f8d3695e8c [model_cards] camembert: dataset = oscar 
Hat/tip @pjox
 2020-10-21 14:17:56 -04:00
Julien Chaumond 758572cad8 [model_cards] move hatmimoha/arabic-ner to correct location 
see https://github.com/huggingface/transformers/commit/16d3cc187ded95946231956460e9004a236474e2 and https://github.com/huggingface/transformers/pull/7836
 2020-10-21 14:13:17 +02:00
Julien Chaumond 99898dcd27 [Pipelines] Fix links to model lists (#7826) 2020-10-16 02:57:02 -04:00
Julien Chaumond e7aa64838c [model_cards] facebook/bart-large-mnli: register ZSC for the inference API 
cc @Narsil @mfuntowicz @joeddav
 2020-10-15 19:02:10 +02:00
Julien Chaumond 6f45dd2fac [model_cards] Fix yaml for Facebook/wmt19-* 
see d99ed7ad61
 2020-10-15 16:14:08 +02:00
Julien Chaumond d99ed7ad61 [model_cards] Facebook: add thumbnail 2020-10-15 12:53:29 +02:00
Julien Chaumond ba654270b3 [model_cards] rename to correct model name 2020-10-14 19:02:48 +02:00
Julien Chaumond c2e0d8ac52 [model_card] nikokons/gpt2-greek 
by @nikkon3
 2020-10-07 16:28:47 -04:00
Julien Chaumond e32390931d [model_card] distilbert-base-german-cased 2020-10-01 09:08:49 -04:00
Julien Chaumond 9a4e163b58 [model_card] Fix metadata, adalbertojunior/PTT5-SMALL-SUM 2020-10-01 08:54:06 -04:00
Julien Chaumond 0acd1ffa09 [doc] rm Azure buttons as not implemented yet 2020-09-30 17:31:08 -04:00
Julien Chaumond eef8d94d19 [model_cards] 
We use ISO 639-1 cc @gentaiscool
 2020-09-18 12:09:24 +02:00
Julien Chaumond d6c08b07a0 [AutoTokenizer] Correct error message 2020-09-10 17:19:01 +02:00
Julien Chaumond ed71c21d6a [from_pretrained] Allow tokenizer_type ≠ model_type (#6995) 2020-09-09 04:22:59 -04:00
Julien Chaumond 10c6f94adc [model_card] register jplu/tf-xlm-r-ner-40-lang as multilingual 2020-09-07 05:03:40 -04:00
Julien Chaumond d4aa7284c8 [model_card] jplu/tf-xlm-r-ner-40-lang: Fix link 
cc @jplu
 2020-09-07 04:33:15 -04:00
Julien Chaumond 5a3aec90a9 [model_card] link to correctly cased piaf dataset 
cc @psorianom @rachelker
 2020-09-03 08:57:32 -04:00
Julien Chaumond d822ab636b [model_cards] Fix file path for flexudy/t5-base-multi-sentence-doctor 2020-09-02 00:02:40 +02:00
Julien Chaumond 3242e4d942 [model_cards] Fix tiny typos 2020-08-26 23:16:06 +02:00
Julien Chaumond 07971d8b18 [model_cards] Fix yaml for cedpsam/chatbot_fr 2020-08-17 21:33:32 +02:00
Julien Chaumond 295466aae6 [model_card] Sample input for rdenadai/BR_BERTo 
cc @rdenadai
 2020-07-24 14:14:10 -04:00
Julien Chaumond 82dd96cae7 [model_cards] Dataset ids are case-sensitive 
cc @lhoestq @thomwolf

Also cc'ing model author @nreimers => Model pages now properly link to the dataset pages (and in the future, eval results, etc.)
 2020-07-20 12:47:28 +02:00
Julien Chaumond 1bca4fbd39 [model_card] Fix metadata 2020-07-17 13:55:37 -04:00
Julien Chaumond 6b6d035d8f [model_card] illuin/lepetit 2020-07-16 03:50:47 -04:00
Julien Chaumond 62ec28ce4f [model_cards] Fix pierreguillou/gpt2-small-portuguese 2020-07-15 22:14:52 +02:00
Julien Chaumond 015dc51fe3 [model_card] bert-portuguese: add language meta 
cc @rodrigonogueira4 @abiocapsouza @robertoalotufo

Also cc @piegu

Obrigado :)
 2020-07-15 21:25:52 +02:00
Julien Chaumond b23d3a5ad4 [model_cards] Switch all languages codes to ISO-639-{1,2,3} 2020-07-15 18:59:20 +02:00
Julien Chaumond 8ab565a4be [model_card] Fix syntax 2020-07-14 22:27:07 +02:00
Julien ChaumondSuraj PatilSylvain Gugger
 		201d23f285 Update The Big Table of Tasks 
Co-Authored-By: Suraj Patil <surajp815@gmail.com>
Co-Authored-By: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-07-10 18:07:29 +02:00
Julien Chaumond bfacb2e34f [model_card] BART for ELI5 
cc @yjernite
 2020-07-10 08:10:24 -04:00
Julien Chaumond 552e4591f5 [model_card] Add meta + fix link to image 
(hotlinking to image works on GitHub but not on external sites)

cc @bashartalafha
 2020-07-10 05:07:33 -04:00
Julien Chaumond 141492448b Create xlm-roberta-large-finetuned-conll03-german-README.md 
cc @BramVanroy
 2020-07-07 13:15:10 -04:00
Julien Chaumond a0a6387a0d [model_cards] roberta-large-mnli: fix sep_token 2020-07-02 10:04:02 -04:00
Julien Chaumond 215db688da Create roberta-large-mnli-README.md 2020-07-02 09:43:54 -04:00
Julien Chaumond c950fef545 [docs] Small tweaks to #5323 2020-06-29 14:24:33 +02:00
Julien Chaumond 6a495cae00 [model_cards] Example of how to specify inputs for the widget 2020-06-25 15:58:25 +02:00
Julien Chaumond fc24a93e64 [HfApi] Add support for pipeline_tag 2020-06-24 16:54:00 +00:00
Julien Chaumond c01480bba3 [file_utils] Type user-agent 2020-06-23 18:31:13 +02:00
Julien ChaumondandSam Shleifer aa6a29bc25 SummarizationPipeline: init required task name (#5086) 
* SummarizationPipeline: init required task name

* Update src/transformers/pipelines.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* Apply suggestions from code review

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-06-20 03:16:30 -04:00
Julien Chaumond 355954ffca Create distilbert-base-uncased-distilled-squad-README.md 2020-06-18 05:17:45 -04:00
Julien Chaumond 70bc3ead4f [TextClassificationPipeline] Hotfix: make json serializable 2020-06-17 15:09:27 +00:00
Julien Chaumond ca5e1cdf8e model_cards: we can now tag datasets 
see corresponding model pages to see how it's rendered
 2020-06-12 23:19:07 +02:00
Julien Chaumond b9109f2de1 [doc] Make it clearer that text-generation does not involve training 2020-06-05 14:59:22 +02:00
Julien Chaumond 99207bd112 Pipelines: miscellanea of QoL improvements and small features... (#4632) 
* [hf_api] Attach all unknown attributes for future-proof compatibility

* [Pipeline] NerPipeline is really a TokenClassificationPipeline

* modelcard.py: I don't think we need to force the download

* Remove config, tokenizer from SUPPORTED_TASKS as we're moving to one model = one weight + one tokenizer

* FillMaskPipeline: also output token in string form

* TextClassificationPipeline: option to return all scores, not just the argmax

* Update docs/source/main_classes/pipelines.rst
 2020-06-03 03:51:31 -04:00
Julien Chaumond b42586ea56 Fix CI after killing archive maps (#4724) 
* 🐛 Fix model ids for BART and Flaubert
 2020-06-02 10:21:09 -04:00
Julien Chaumond d4c2cb402d Kill model archive maps (#4636) 
* Kill model archive maps

* Fixup

* Also kill model_archive_map for MaskedBertPreTrainedModel

* Unhook config_archive_map

* Tokenizers: align with model id changes

* make style && make quality

* Fix CI
 2020-06-02 09:39:33 -04:00
Julien Chaumond ec8717d5d8 [config] Ensure that id2label always takes precedence over num_labels 2020-06-01 16:54:55 +02:00
Julien Chaumond 751a1e0890 [config] Ensure that id2label always takes precedence over num_labels 
Fixes bug reported in https://github.com/huggingface/transformers/issues/4669

See #3967 for context
 2020-06-01 16:25:56 +02:00
Julien Chaumond 365d452d4d [ci] Slow GPU tests run daily (#4465) 2020-05-25 17:28:02 -04:00
Julien Chaumond 2c1ebb8b50 Re-apply #4446 + add packaging dependency 
As discussed w/ @lysandrejik

packaging is maintained by PyPA (the Python Packaging Authority), and should be lightweight and stable
 2020-05-22 17:29:03 -04:00
Julien Chaumond f7677e1623 [model_cards] bart-large-cnn 
cc @sshleifer
 2020-05-22 12:20:54 -04:00
Julien Chaumond 865d4d595e [ci] Close #4481 2020-05-20 18:27:42 -04:00
Julien Chaumond a3af8e86cb Update test_trainer_distributed.py 2020-05-20 18:26:51 -04:00
Julien Chaumond bf14ef75f1 [Trainer] move model to device before setting optimizer (#4450) 2020-05-18 23:13:33 -04:00
Julien Chaumond 5e7fe8b585 Distributed eval: SequentialDistributedSampler + gather all results (#4243) 
* Distributed eval: SequentialDistributedSampler + gather all results

* For consistency only write to disk from world_master

Close https://github.com/huggingface/transformers/issues/4272

* Working distributed eval

* Hook into scripts

* Fix #3721 again

* TPU.mesh_reduce: stay in tensor space

Thanks @jysohn23

* Just a small comment

* whitespace

* torch.hub: pip install packaging

* Add test scenarii
 2020-05-18 22:02:39 -04:00
Julien Chaumond 4c06893610 Fix nn.DataParallel compatibility in PyTorch 1.5 (#4300) 
* Test case for #3936

* multigpu tests pass on pytorch 1.4.0

* Fixup

* multigpu tests pass on pytorch 1.5.0

* Update src/transformers/modeling_utils.py

* Update src/transformers/modeling_utils.py

* rename multigpu to require_multigpu

* mode doc
 2020-05-18 20:34:50 -04:00
Julien Chaumond 757baee846 Fix un-prefixed f-string 
see https://github.com/huggingface/transformers/pull/4367#discussion_r426356693

Hat/tip @girishponkiya
 2020-05-18 11:20:46 -04:00
Julien Chaumond 3e0f062106 Fix addcmul_ 2020-05-15 17:44:17 -04:00
Julien Chaumond fc2a4c88ce Fix: one more try 2020-05-15 17:38:48 -04:00
Julien Chaumond 55bda52555 Same fix for addcmul_ 2020-05-15 17:23:48 -04:00
Julien Chaumond ad02c961c6 Fix UserWarning: This overload of add_ is deprecated in pytorch==1.5.0 2020-05-15 17:09:11 -04:00
Julien Chaumond 15550ce0d1 [skip ci] remove local rank 2020-05-15 17:08:38 -04:00
Julien Chaumond af2e6bf87c [examples] Streamline doc 2020-05-14 20:34:31 -04:00
Julien Chaumond 448c467256 Fix: unpin flake8 and fix cs errors (#4367) 
* Fix: unpin flake8 and fix cs errors

* Ok we still need to quote those
 2020-05-14 13:14:26 -04:00
Julien Chaumond c547f15a17 Use Filelock to ensure distributed barriers 
see context in https://github.com/huggingface/transformers/pull/4223
 2020-05-14 11:58:32 -04:00
Julien Chaumond 015f7812ed [ci skip] Pin isort 2020-05-14 10:12:18 -04:00
Julien ChaumondChris Van PeltBoris Dayma
 		241759101e (v2) Improvements to the wandb integration (#4324) 
* Improvements to the wandb integration

* small reorg + no global necessary

* feat(trainer): log epoch and final metrics

* Simplify logging a bit

* Fixup

* Fix crash when just running eval

Co-authored-by: Chris Van Pelt <vanpelt@gmail.com>
Co-authored-by: Boris Dayma <boris.dayma@gmail.com>
 2020-05-12 21:52:01 -04:00
Julien Chaumond 4bf5042240 Fix BART tests on GPU (#4298) 2020-05-12 09:11:50 -04:00
Julien Chaumond 56e8ef632f [ci] Restrict GPU tests to actual code commits 2020-05-11 20:40:41 -04:00
Julien Chaumond ba6f6e44a8 [ci] Re-enable torch GPU tests 2020-05-12 00:05:36 +00:00
Julien Chaumond 7b75aa9fa5 [TPU] Doc, fix xla_spawn.py, only preprocess dataset once (#4223) 
* [TPU] Doc, fix xla_spawn.py, only preprocess dataset once

* Update examples/README.md

* [xla_spawn] Add `_mp_fn` to other Trainer scripts

* [TPU] Fix: eval dataloader was None
 2020-05-08 14:10:05 -04:00
Julien Chaumond 274d850d34 Fix #4098 2020-05-08 12:39:46 -04:00
Julien Chaumond c99fe0386b [doc] Fix broken links + remove crazy big notebook 2020-05-07 18:44:18 -04:00
Julien Chaumond 6669915b65 [examples] Add column for pytorch-lightning support 2020-05-07 15:26:58 -04:00
Julien Chaumond 612fa1b10b Examples readme.md (#4215) 
* README

* Update README.md
 2020-05-07 15:00:06 -04:00
Julien Chaumond 0ae96ff8a7 BIG Reorganize examples (#4213) 
* Created using Colaboratory

* [examples] reorganize files

* remove run_tpu_glue.py as superseded by TPU support in Trainer

* Bugfix: int, not tuple

* move files around
 2020-05-07 13:48:44 -04:00
Julien Chaumond cafa6a9e29 [Trainer] Ability to specify optimizer/scheduler at init 
cc @patrickvonplaten @thomwolf
 2020-05-07 11:25:26 -04:00
Julien Chaumond fd2174664c [Trainer] W&B: Enable model watch 
See https://github.com/huggingface/transformers/pull/3916
 2020-05-05 10:59:23 -04:00
Julien Chaumond d713cfc5eb GePpeTto 🇮🇹: Fixpath to model card 2020-05-01 11:48:58 -04:00
Julien Chaumond 27d55125e6 Configs: saner num_labels in configs. (#3967) 2020-05-01 11:28:55 -04:00
Julien Chaumond b8686174be Merge pull request #3934 from huggingface/examples_args_from_files 
[qol] example scripts: parse args from .args file or JSON
 2020-04-30 22:40:13 -04:00
Julien Chaumond f39217a5ec [tests] Light cleanup of tempfile in tests/ 2020-04-30 22:30:15 -04:00
Julien Chaumond f54dc3f4d5 [ci] Load pretrained models into the default (long-lived) cache 
There's an inconsistency right now where:
- we load some models into CACHE_DIR
- and some models in the default cache
- and often, in both for the same models

When running the RUN_SLOW tests, this takes a lot of disk space, time, and bandwidth.

I'd rather always use the default cache
 2020-04-30 22:30:15 -04:00
Julien Chaumond ab90353f1a [cli] {login, upload, s3} display more helpful error messages 2020-04-30 12:51:06 -04:00
Julien Chaumond 452dd0e4d9 [ci] Align test_hf_api.py with API change 2020-04-30 12:06:01 -04:00
Julien Chaumond 6faca88ee0 Align MarianMT with #4030 
cc @sshleifer
 2020-04-28 20:35:20 -04:00
Julien Chaumond 211e130811 [github] Issue templates: populate some labels 
cc @bramvanroy @stefan-it
 2020-04-28 20:34:34 -04:00
Julien Chaumond 455c639093 CDN urls (#4030) 
* [file_utils] use_cdn + documentation

* Move to cdn. urls for weights

* [urls] Hotfix for bert-base-japanese
 2020-04-28 20:27:14 -04:00
Julien Chaumond 97a375484c rm boto3 dependency 2020-04-27 11:17:14 -04:00
Julien Chaumond c811526004 [examples] For convenience, also save the tokenizer 
Close #3921
 2020-04-24 09:52:42 -04:00