c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,208 Commits 171 Branches 49 Tags
00a1fc9ae4ae3083bdedd76df6fddbd6894d0aed
Commit Graph
5208 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Quentin Lhoest 00a1fc9ae4 pass config to AutoTokenizer.from_pretrained for Rag tokenizers 2020-09-15 13:00:00 +02:00
Patrick von Platen 2faaa4ad3c Merge branch 'finalize_rag' of https://github.com/huggingface/transformers into finalize_rag 2020-09-15 12:21:54 +02:00
Patrick von Platen c1bc9fe05d delete unnecessary imports 2020-09-15 12:21:25 +02:00
Quentin Lhoest 6e9f30748f don't save paths 2020-09-15 12:14:36 +02:00
Quentin Lhoest bc440f3e7c add RagTokenizer.save/from_pretrained and RagRetriever.save/from_pretrained 2020-09-15 11:44:54 +02:00
Patrick von Platen 2094d37888 make all integration tests pass 2020-09-15 01:23:05 +02:00
Patrick von Platen b3f9e986d9 save working solution 2020-09-15 00:36:34 +02:00
Patrick von Platen b4a094dd97 add from encoder generator 2020-09-14 21:23:25 +02:00
Patrick von Platen 3688823d19 change structure 2020-09-14 21:00:26 +02:00
Patrick von Platen e0a37450e1 change structure 2020-09-14 18:32:14 +02:00
Patrick von Platen 720ee41342 add tests; fix position ids warning 2020-09-14 13:21:55 +02:00
Patrick von Platen 17a8621661 add tokenizer to slow integration test and allow retriever to run on cpu 2020-09-14 12:37:43 +02:00
Your Name 64796004dc RAG integration tests 2020-09-13 16:32:23 -07:00
Your Name df5eec9f14 rename nlp->datasets 2020-09-12 14:19:43 -07:00
Your Name 1497120d89 Post-rebase refactor 2020-09-12 14:16:53 -07:00
Your Name 593765088e Merge remote-tracking branch 'upstream/master' into rag 2020-09-12 14:00:57 -07:00
Your Name 4ad9cb1bd9 Fix tests 3 2020-09-12 12:37:46 -07:00
Your Name 6215f3b5e0 Refactor test requirements 2020-09-12 12:16:34 -07:00
Your Name 9795dc3464 Fix tests 2 2020-09-12 11:37:29 -07:00
Your Name a4c7c25cd2 Fix tests 2020-09-12 11:20:30 -07:00
Your Name 945d56995e Extra dependencied for tests 2020-09-12 11:12:26 -07:00
Your Name e82aca09a5 Improve readme + post-rebase refactor 2020-09-12 11:07:18 -07:00
Sam Shleifer b76cb1c3df [s2s] run_eval supports --prefix clarg. (#6953) 2020-09-12 01:08:21 -04:00
李明浩 563ffb3dc3 Create README.md (#7066) 2020-09-11 15:21:05 -04:00
李明浩 1ad49cde3a Create README.md (#7067) 2020-09-11 15:20:54 -04:00
Sagor SarkerandJulien Chaumond 4753816e39 added bangla-bert-base model card and also modified other model cards (#7071) 
* added bangla-bert-base

* Apply suggestions from code review

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-09-11 15:17:25 -04:00
Suraj Patil 0a8c17d53c [T5Tokenizer] remove prefix_tokens (#7078) 2020-09-11 14:18:45 -04:00
Sylvain Gugger 4cbd50e611 Compute loss method (#7074) 2020-09-11 12:06:31 -04:00
Sylvain GuggerandPatrick von Platen ae736163d0 Add tests and fix various bugs in ModelOutput (#7073) 
* Add tests and fix various bugs in ModelOutput

* Update tests/test_model_output.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-09-11 12:01:33 -04:00
Sylvain Gugger e841b75dec Automate the lists in auto-xxx docs (#7061) 
* More readable dict

* More nlp -> datasets

* Revert "More nlp -> datasets"

This reverts commit 3cd1883d226c63c4a686fc1fed35f2cd586ebe45.

* Automate the lists in auto-xxx docs

* More readable dict

* Revert "More nlp -> datasets"

This reverts commit 3cd1883d226c63c4a686fc1fed35f2cd586ebe45.

* Automate the lists in auto-xxx docs

* nlp -> datasets

* Fix new key
 2020-09-11 10:42:09 -04:00
Sylvain Gugger 0054a48cdd Add dep on datasets (#7058) 2020-09-11 04:43:19 -04:00
Patrick von Platen 221d4c63a3 clean naming (#7068) 2020-09-11 09:57:53 +02:00
Stas Bekman 8fcbe486e1 these tests require non-multigpu env (#7059) 
* these tests require non-multigpu env

* cleanup

* clarify
 2020-09-10 18:52:55 -04:00
Sam Shleifer 77950c485a [wip/s2s] DistributedSortishSampler (#7056) 2020-09-10 15:23:44 -04:00
Sylvain Gugger 514486739c Fix CI with change of name of nlp (#7054) 
* nlp -> datasets

* More nlp -> datasets

* Woopsie

* More nlp -> datasets

* One last
 2020-09-10 14:51:08 -04:00
Sam Shleifer e9a2f772bc [s2s] --eval_max_generate_length (#7018) 2020-09-10 14:11:34 -04:00
Stas Bekman df4594a9da [xlm tok] config dict: fix str into int to match definition (#7034) 2020-09-10 19:31:01 +02:00
Julien Chaumond d6c08b07a0 [AutoTokenizer] Correct error message 2020-09-10 17:19:01 +02:00
Patrick von Platen db38f7ce29 [BertGeneration, Docs] Fix another old name in docs (#7050) 
* correct docs for bert generation

* upload
 2020-09-10 17:12:33 +02:00
Patrick von Platen 3bd95b0faf correct docs for bert generation (#7048) 2020-09-10 17:08:40 +02:00
Patrick von Platen eb2feb5d90 Create README.md 2020-09-10 17:05:50 +02:00
Ashwin Geet DsaandAshwin Geet Dsa 66a5a6fda8 fix to ensure that returned tensors after the tokenization is Long (#7039) 
* fix to ensure that returned tensors after the tokenization is Long

* fix to ensure that returned tensors after the tokenization is Long

Co-authored-by: Ashwin Geet Dsa <adsa@grvingt-6.nancy.grid5000.fr>
 2020-09-10 11:04:03 -04:00
Patrick von Platen 9ccdb1d517 Update README.md 2020-09-10 17:01:19 +02:00
Patrick von Platen 60698936fc Create README.md 2020-09-10 17:00:10 +02:00
Patrick von Platen e0c3bc8ee0 Create README.md 2020-09-10 16:51:15 +02:00
Patrick von Platen c356b9878d Create README.md 2020-09-10 16:45:44 +02:00
Patrick von Platen 5afd3f6196 Create README.md 2020-09-10 16:44:47 +02:00
Sylvain GuggerandLysandre Debut 15a189049e Add TF Funnel Transformer (#7029) 
* Add TF Funnel Transformer

* Proper dummy input

* Formatting

* Update src/transformers/modeling_tf_funnel.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Address review comments

* One review comment forgotten

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-09-10 10:41:56 -04:00
Patrick von Platen 7fd1febf38 Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. (#6594) 
* add conversion script

* improve conversion script

* make style

* add tryout files

* fix

* update

* add causal bert

* better names

* add tokenizer file as well

* finish causal_bert

* fix small bugs

* improve generate

* change naming

* renaming

* renaming

* renaming

* remove leftover files

* clean files

* add fix tokenizer

* finalize

* correct slow test

* update docs

* small fixes

* fix link

* adapt check repo

* apply sams and sylvains recommendations

* fix import

* implement Lysandres recommendations

* fix logger warn
 2020-09-10 16:40:51 +02:00
Sylvain Gugger d1691d90e5 Samell fixed in tf template (#7044) 2020-09-10 10:36:02 -04:00