c-my/transformers
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,820 Commits 171 Branches 49 Tags
aec51e56960545ae3ee192f49524653ad27343c4
Commit Graph
5820 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lysandre aec51e5696 v3.5.0 documentation 2020-11-10 08:58:47 -05:00
Lysandre 818878dc88 Release: v3.5.0 v3.5.0 2020-11-10 08:50:43 -05:00
Lysandre Debut 9cebee38ad Model sharing rst (#8439) 
* Update RST

* Finer details

* Re-organize

* Style
 2020-11-10 08:35:11 -05:00
Julien Chaumond ad2303a401 Fix style 2020-11-10 14:28:30 +01:00
Julien Chaumond 55e8d0cea2 Update links from s3 to huggingface.co 2020-11-10 14:03:29 +01:00
Lysandre DebutandNicolas Patry 850afb422d Patch token classification pipeline (#8364) 
* Patch token classification pipeline

* Some added tests for TokenClassificationArgumentHandler (#8366)

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
 2020-11-10 07:29:34 -05:00
Julien ChaumondSylvain GuggerSam Shleifer
 		70f622fab4 Model versioning (#8324) 
* fix typo

* rm use_cdn & references, and implement new hf_bucket_url

* I'm pretty sure we don't need to `read` this file

* same here

* [BIG] file_utils.networking: do not gobble up errors anymore

* Fix CI 😇

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Tiny doc tweak

* Add doc + pass kwarg everywhere

* Add more tests and explain

cc @sshleifer let me know if better

Co-Authored-By: Sam Shleifer <sshleifer@gmail.com>

* Also implement revision in pipelines

In the case where we're passing a task name or a string model identifier

* Fix CI 😇

* Fix CI

* [hf_api] new methods + command line implem

* make style

* Final endpoints post-migration

* Fix post-migration

* Py3.6 compat

cc @stefan-it

Thank you @stas00

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-11-10 07:11:02 -05:00
Teven 4185b115d4 Changing XLNet default from not using memories to 512 context size following paper (#8417) 
* Move XLNet memory length FutureWarning

* isort

* style

* Changed default XLNet memory length
 2020-11-09 20:49:51 -05:00
Stas BekmanandSam Shleifer 190df58560 [github CI] add a multi-gpu job for all example tests (#8341) 
* add a multi-gpu job for all example tests

* run only ported tests

* rename

* explain why env is re-activated on each step

* mark all unported/checked tests with @require_torch_non_multigpu_but_fix_me

* style

* Apply suggestions from code review

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-11-09 15:47:38 -05:00
Sylvain Gugger a39218b75b Check all models are in an auto class (#8425) 2020-11-09 15:44:54 -05:00
Stas BekmanandLysandre Debut ef032ddd1e [docs] [testing] gpu decorators table (#8422) 
* gpu decorators table

* whitespace

* Update docs/source/testing.rst

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* whitespace

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-11-09 14:27:42 -05:00
Sam Shleifer a8339b9ecc Fix bart shape comment (#8423) 2020-11-09 13:25:33 -05:00
Sam Shleifer 46509d1c19 [docs] remove sshleifer from issue-template :( (#8418) 2020-11-09 12:51:38 -05:00
Patrick von Platen 9c83b96e62 [Tests] Add Common Test for Training + Fix a couple of bugs (#8415) 
* add training tests

* correct longformer

* fix docs

* fix some tests

* fix some more train tests

* remove ipdb

* fix multiple edge case model training

* fix funnel and prophetnet

* clean gpt models

* undo renaming of albert
 2020-11-09 18:24:41 +01:00
Sylvain Gugger 52040517b8 Deprecate old data/metrics functions (#8420) 2020-11-09 12:10:09 -05:00
Stas Bekman d4d1fbfc5a [fsmt convert script] fairseq broke chkpt data - fixing that (#8377) 
* fairseq broke chkpt data - fixing that

* style

* support older bpecodes filenames - specifically "code" in iwslt14
 2020-11-09 11:57:42 -05:00
Sylvain Gugger 5c766ecb50 Fix typo 2020-11-09 11:50:51 -05:00
Sylvain GuggerandThomas Wolf 908a28894c Add new token classification example (#8340) 
* Add new token classification example

* Remove txt file

* Add test

* With actual testing done

* Less warmup is better

* Update examples/token-classification/run_ner_new.py

Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

* Address review comments

* Fix test

* Make Lysandre happy

* Last touches and rename

* Rename in tests

* Address review comments

* More run_ner -> run_ner_old

Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>
 2020-11-09 11:39:55 -05:00
Sylvain Gugger c7cb1aa26c Bump tokenizers (#8419) 2020-11-09 11:32:10 -05:00
Stas Bekman 78d706f3ae [fsmt tokenizer] support lowercase tokenizer (#8389) 
* support lowercase tokenizer

* fix arg pos
 2020-11-09 10:41:39 -05:00
Shashank Gupta 1e2acd0dcf Bug fix for permutation language modelling (#8409) 2020-11-09 10:23:26 -05:00
Philip MayandSylvain Gugger bf8625e70b add evaluate doc - trainer.evaluate returns 'epoch' from training (#8273) 
* add evaluate doc

* fix style with utils/style.doc

* Update src/transformers/trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-11-09 09:00:59 -05:00
Sam Shleifer ebde57acac examples/docs: caveat that PL examples don't work on TPU (#8309) 2020-11-09 08:55:22 -05:00
Julien Plu 76e7a44dee Fix some tooling for windows (#8359) 
* Fix some tooling for windows

* Fix conflict

* Trigger CI
 2020-11-09 13:50:38 +01:00
dartrevan 507dfb40c3 Update README.md (#8406) 2020-11-09 16:44:43 +08:00
smanjil 7247d0b4ea updating tag for exbert viz (#8408) 2020-11-09 16:43:55 +08:00
Stas Bekman 4ab5617b0b comet_ml temporary fix(#8410) 2020-11-09 16:36:06 +08:00
Sam Shleifer e6d9cdaafe [s2s/distill] remove run_distiller.sh, fix xsum script (#8412) 2020-11-08 16:57:43 -05:00
Stas Bekman 66582492d3 [s2s test_finetune_trainer] failing multigpu test (#8400) 2020-11-08 16:45:40 -05:00
Stas Bekman f62755a600 [s2s examples test] fix data path (#8398) 2020-11-08 16:44:18 -05:00
Jonathan Chang 4a53e8e9e4 Fix DataCollatorForWholeWordMask again (#8397) 2020-11-08 09:53:01 -05:00
Manav Rathod 610730998f fixed default labels for QA model (#8399) 2020-11-08 09:08:14 -05:00
Chengxi GuoandJulien Chaumond 0b02489b2c Add gpt2-medium-chinese model card (#8402) 
* Create README.md

* Update model_cards/mymusise/gpt2-medium-chinese/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-08 05:00:19 -05:00
Stas Bekman 187554366f fix md table (#8395) 2020-11-08 04:25:14 -05:00
Jonathan Chang 77a257fc21 Fix DataCollatorForWholeWordMask (#8379) 
* Fix DataCollatorForWholeWordMask

* Replace all tensorize_batch in data_collator.py
 2020-11-07 12:51:56 -05:00
Stas Bekman 517eaf460b [make] rewrite modified_py_files in python to be cross-platform (#8371) 
* rewrite modified_py_files in python to be cross-platform

* try a different way to test for variable not being ""

* improve comment
 2020-11-07 18:45:16 +01:00
Patrick von Platen 07708793f2 fix encoder outputs (#8368) 2020-11-06 21:03:25 +01:00
Yossi SynettandPatrick von Platen bc0d26d1de [All Seq2Seq model + CLM models that can be used with EncoderDecoder] Add cross-attention weights to outputs (#8071) 
* Output cross-attention with decoder attention output

* Update src/transformers/modeling_bert.py

* add cross-attention for t5 and bart as well

* fix tests

* correct typo in docs

* add sylvains and sams comments

* correct typo

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-11-06 19:34:48 +01:00
hassoudi 30f2507a07 Update README.md (#8360) 
Fix websitr address
 2020-11-06 11:45:46 -05:00
Jonathan Chang 5807ba3fa9 Fix typo (#8351) 2020-11-06 11:19:41 -05:00
hassoudi 82146496b6 Update README.md (#8338) 
fixes
 2020-11-06 06:20:58 -05:00
ktrapeznikovandJulien Chaumond 9e5c4d39ab Create README.md (#8312) 
* Create README.md

* Update model_cards/ktrapeznikov/gpt2-medium-topic-news/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 06:19:59 -05:00
hasantanvir79andJulien Chaumond 06ebc37967 Create README.md (#8255) 
* Create README.md

Initial commit

* Updated Read me

Updated

* Apply suggestions from code review

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 03:34:24 -05:00
Karthik Uppuluri 41cd031cf2 Create README.md (#8169) 2020-11-06 03:26:07 -05:00
Karthik Uppuluri f932ddeff5 Create README.md (#8170) 2020-11-06 03:25:52 -05:00
Karthik Uppuluri 08b92f78fa Create README.md (#8168) 
* Create README.md

* Update README.md
 2020-11-06 03:25:33 -05:00
Karthik UppuluriandJulien Chaumond 77d62e78b0 Create README.md (#8167) 
* Create README.md

Telugu BERTU Readme file

* Update model_cards/kuppuluri/telugu_bertu/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 03:24:31 -05:00
Yifan Peng dd6bfcaefb Create README.md (#8327) 2020-11-06 03:22:52 -05:00
smanjilandJulien Chaumond ddeecf08e6 german medbert model details (#8266) 
* model details

* Apply suggestions from code review

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 03:21:13 -05:00
Jiaxin Pei 96baaafd34 Create README.md (#8258) 2020-11-06 03:19:12 -05:00