c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,829 Commits 171 Branches 49 Tags
1851a64b6f19c2c034c2ca9a0e7df5a382b76913
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick von Platen 9a0a8c1c6f add examples to doc (#4045) 2020-04-28 16:33:23 +02:00
Patrick von Platen fa49b9afea Clean Encoder-Decoder models with Bart/T5-like API and add generate possibility (#3383) 
* change encoder decoder style to bart & t5 style

* make encoder decoder generation dummy work for bert

* make style

* clean init config in encoder decoder

* add tests for encoder decoder models

* refactor and add last tests

* refactor and add last tests

* fix attn masks for bert encoder decoder

* make style

* refactor prepare inputs for Bert

* refactor

* finish encoder decoder

* correct typo

* add docstring to config

* finish

* add tests

* better naming

* make style

* fix flake8

* clean docstring

* make style

* rename
 2020-04-28 15:11:09 +02:00
Patrick von Platen 180585741c [Generation] Generation should allow to start with empty prompt (#3993) 
* fix empty prompt

* fix length in generation pipeline
 2020-04-28 14:33:15 +02:00
Patrick von Platen 52679fbc2e add dialogpt training tips (#3996) 2020-04-28 14:32:31 +02:00
Patrick von Platen c4158a6314 [Pipelines] Encode to max length of input not max length of tokenizer for batch input (#3857) 
* remove max_length = tokenizer.max_length when encoding

* make style
 2020-04-20 14:39:16 -04:00
Patrick von Platen a21d4fa410 add "by" to ReadMe 2020-04-18 18:07:17 +02:00
Patrick von Platen e9d0bc027a [Config, Serialization] more readable config serialization (#3797) 
* better config serialization

* finish configuration utils
 2020-04-17 20:07:18 -04:00
Patrick von Platen 1d4a35b396 Higher tolerance for past testing in TF T5 (#3844) 2020-04-17 11:26:16 -04:00
Patrick von Platen d13eca11e2 Higher tolerance for past testing in T5 (#3843) 2020-04-17 11:25:14 -04:00
Patrick von Platen f399c00610 Update README 2020-04-17 09:42:22 +02:00
Patrick von PlatenandThomas Liao 80a1694514 [Examples, T5] Change newstest2013 to newstest2014 and clean up (#3817) 
* Refactored use of newstest2013 to newstest2014. Fixed bug where argparse consumed first command line argument as model_size argument rather than using default model_size by forcing explicit --model_size flag inclusion

* More pythonic file handling through 'with' context

* COSMETIC - ran Black and isort

* Fixed reference to number of lines in newstest2014

* Fixed failing test. More pythonic file handling

* finish PR from tholiao

* remove outcommented lines

* make style

* make isort happy

Co-authored-by: Thomas Liao <tholiao@gmail.com>
 2020-04-16 20:00:41 +02:00
Patrick von Platen baca8fa8e6 clean pipelines (#3795) 2020-04-16 10:21:34 -04:00
Patrick von Platen 38f7461df3 [TFT5, Cache] Add cache to TFT5 (#3772) 
* correct gpt2 test inputs

* make style

* delete modeling_gpt2 change in test file

* translate from pytorch

* correct tests

* fix conflicts

* fix conflicts

* fix conflicts

* fix conflicts

* make tensorflow t5 caching work

* make style

* clean reorder cache

* remove unnecessary spaces

* fix test
 2020-04-16 16:14:52 +02:00
Patrick von Platen a5b249472e change pad token id to config pad token id (#3793) 2020-04-16 15:58:57 +02:00
Patrick von PlatenandJulien Chaumond d22894dfd4 [Docs] Add DialoGPT (#3755) 
* add dialoGPT

* update README.md

* fix conflict

* update readme

* add code links to docs

* Update README.md

* Update dialo_gpt2.rst

* Update pretrained_models.rst

* Update docs/source/model_doc/dialo_gpt2.rst

Co-Authored-By: Julien Chaumond <chaumond@gmail.com>

* change filename of dialogpt

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-04-16 09:04:32 +02:00
Patrick von Platen 301bf8d1b4 Create Modelcard for Reformer Model 2020-04-15 16:26:24 +02:00
Patrick von Platen 01c37dcdb5 [Config, Caching] Remove output_past everywhere and replace by use_cache argument (#3734) 
* remove output_past from pt

* make style

* add optional input length for gpt2

* add use cache to prepare input

* save memory in gpt2

* correct gpt2 test inputs

* make past input optional for gpt2

* finish use_cache for all models

* make style

* delete modeling_gpt2 change in test file

* correct docstring

* correct is true statements for gpt2
 2020-04-14 14:40:28 -04:00
Patrick von Platen 092cf881a5 [Generation, EncoderDecoder] Apply Encoder Decoder 1.5GB memory… (#3778) 2020-04-13 22:29:28 -04:00
Patrick von PlatenandYacine Jernite ce2298fb5f [T5, generation] Add decoder caching for T5 (#3682) 
* initial commit to add decoder caching for T5

* better naming for caching

* finish T5 decoder caching

* correct test

* added extensive past testing for T5

* clean files

* make tests cleaner

* improve docstring

* improve docstring

* better reorder cache

* make style

* Update src/transformers/modeling_t5.py

Co-Authored-By: Yacine Jernite <yjernite@users.noreply.github.com>

* make set output past work for all layers

* improve docstring

* improve docstring

Co-authored-by: Yacine Jernite <yjernite@users.noreply.github.com>
 2020-04-10 01:02:50 +02:00
Patrick von Platen b0ad069517 [Tokenization] fix edge case for bert tokenization (#3517) 
* fix egde gase for bert tokenization

* add Lysandres comments for improvement

* use new is_pretokenized_flag
 2020-04-07 16:26:31 -04:00
Patrick von Platen 80fa0f7812 [Examples, Benchmark] Improve benchmark utils (#3674) 
* improve and add features to benchmark utils

* update benchmark style

* remove output files
 2020-04-07 16:25:57 -04:00
Patrick von Platen 2ee410560e [Generate, Test] Split generate test function into beam search, no beam search (#3601) 
* split beam search and no beam search test

* fix test

* clean generate tests
 2020-04-06 10:37:05 +02:00
Patrick von Platen 1789c7daf1 fix argument order (#3637) 2020-04-05 12:33:41 +02:00
Patrick von Platen b809d2f073 Fix TF T5 docstring (#3636) 2020-04-05 12:23:09 +02:00
Patrick von Platen f68d22850c delete bogus print statement (#3595) 2020-04-02 21:49:34 +02:00
Patrick von Platen 390c128592 [Encoder-Decoder] Force models outputs to always have batch_size as their first dim (#3536) 
* solve conflicts

* improve comments
 2020-04-02 15:18:33 +02:00
Patrick von Platen ab5d06a094 [T5, examples] replace heavy t5 models with tiny random models (#3556) 
* replace heavy t5 models with tiny random models as was done by sshleifer

* fix isort
 2020-04-02 12:34:05 +02:00
Patrick von Platen a4ee4da18a [T5, TF 2.2] change tf t5 argument naming (#3547) 
* change tf t5 argument naming for TF 2.2

* correct bug in testing
 2020-04-01 22:04:20 +02:00
Patrick von Platen 06dd597552 fix bug in warnings T5 pipelines (#3545) 2020-04-01 21:59:12 +02:00
Patrick von Platen b815edf69f [T5, Testst] Add extensive hard-coded integration tests and make sure PT and TF give equal results (#3550) 
* add some t5 integration tests

* finish summarization and translation integration tests for T5 - results loook good

* add tf test

* fix == vs is bug

* fix tf beam search error and make tf t5 tests pass
 2020-04-01 18:01:33 +02:00
Patrick von Platen b38d552a92 [Generate] Add bad words list argument to the generate function (#3367) 
* add bad words list

* make style

* add bad_words_tokens

* make style

* better naming

* make style

* fix typo
 2020-03-31 18:42:31 +02:00
Patrick von Platen ae6834e028 [Examples] Clean summarization and translation example testing files for T5 and Bart (#3514) 
* fix conflicts

* add model size argument to summarization

* correct wrong import

* fix isort

* correct imports

* other isort make style

* make style
 2020-03-31 17:54:13 +02:00
Patrick von Platen 83d1fbcff6 [Docs] Add usage examples for translation and summarization (#3538) 2020-03-31 09:36:03 -04:00
Patrick von Platen 55bcae7f25 remove useless and confusing lm_labels line (#3531) 2020-03-31 09:32:25 -04:00
Patrick von Platen 42e1e3c67f Update usage doc regarding generate fn (#3504) 2020-03-31 09:31:46 -04:00
Patrick von Platen 57b0fab692 Add better explanation to check docs locally. (#3459) 2020-03-31 09:30:17 -04:00
Patrick von Platen 296252c49e fix lm lables in docstring (#3529) 2020-03-30 14:26:24 +02:00
Patrick von Platen 75ec6c9e3a [T5] make decoder input ids optional for t5 training (#3521) 
* make decoder input ids optional for t5 training

* lm_lables should not be shifted in t5

* add tests

* finish shift right functionality for PT T5

* move shift right to correct class

* cleaner code

* replace -100 values with pad token id

* add assert statement

* remove unnecessary for loop

* make style
 2020-03-30 13:45:26 +02:00
Patrick von Platen 5b44e0a31b [T5] Add training documenation (#3507) 
* Add clear description of how to train T5

* correct docstring in T5

* correct typo

* correct docstring format

* update t5 model docs

* implement collins feedback

* fix typo and add more explanation for sentinal tokens

* delete unnecessary todos
 2020-03-30 13:35:53 +02:00
Patrick von Platen 17dceae7a1 Fix circle ci flaky fail of wmt example (#3485) 
* force bleu

* fix wrong file name

* rename file

* different filenames for each example test

* test files should clean up after themselves

* test files should clean up after themselves

* do not force bleu

* correct typo

* fix isort
 2020-03-27 13:01:28 -04:00
Patrick von Platen 00ea100e96 add summarization and translation to notebook (#3478) 2020-03-27 11:05:37 -04:00
Patrick von Platen f4f4946836 Rename t5-large to t5-base in README.md 2020-03-27 15:57:58 +01:00
Patrick von Platen fa9af2468a Add T5 to docs (#3461) 
* add t5 docs basis

* improve docs

* add t5 docs

* improve t5 docstring

* add t5 tokenizer docstring

* finish docstring

* make style

* add pretrained models

* correct typo

* make examples work

* finalize docs
 2020-03-27 10:57:16 -04:00
Patrick von Platen 5ad2ea06af Add wmt translation example (#3428) 
* add translation example

* make style

* adapt docstring

* add gpu device as input for example

* small renaming

* better README
 2020-03-26 19:07:59 +01:00
Patrick von Platen b4fb94fe6d revert unpin isort commit 2020-03-26 13:19:18 -04:00
Patrick von Platen e703e923ca Add t5 summarization example (#3411) 
* rebase to master

* change tf to pytorch

* change to pytorch

* small fix

* renaming

* add gpu training possibility

* renaming

* improve README

* incoorporate collins feedback

* better Readme

* better README.md
 2020-03-26 18:17:55 +01:00
Patrick von Platen 311970546f rename string in pipeline 2020-03-26 14:59:49 +01:00
Patrick von Platen 022e8fab97 Adds translation pipeline (#3419) 
* fix merge conflicts

* add t5 summarization example

* change parameters for t5 summarization

* make style

* add first code snippet for translation

* only add prefixes

* add prefix patterns

* make style

* renaming

* fix conflicts

* remove unused patterns

* solve conflicts

* fix merge conflicts

* remove translation example

* remove summarization example

* make sure tensors are in numpy for float comparsion

* re-add t5 config

* fix t5 import config typo

* make style

* remove unused numpy statements

* update doctstring

* import translation pipeline
 2020-03-26 13:50:58 +01:00
Patrick von Platen 9c683ef01e Add t5 to pipeline(task='summarization') (#3413) 
* solve conflicts

* move warnings below

* incorporate changes

* add pad_to_max_length to pipelines

* add bug fix for T5 beam search

* add prefix patterns

* make style

* fix conflicts

* adapt pipelines for task specific parameters

* improve docstring

* remove unused patterns
 2020-03-26 11:03:13 +01:00
Patrick von Platen ffa17fe322 Extend config with task specific configs. (#3433) 
* add new default configs

* change prefix default to None
 2020-03-25 21:32:04 +01:00
Patrick von Platen e392ba6938 Add camembert integration tests (#3375) 
* add integration tests for camembert

* use jplu/tf-camembert fro the moment

* make style
 2020-03-24 10:18:37 +01:00
Patrick von Platen 95e00d0808 Clean special token init in modeling_....py (#3264) 
* make style

* fix conflicts
 2020-03-20 21:41:04 +01:00
Patrick von PlatenandMorgan Funtowicz bbf26c4e61 Support T5 Generation (#3228) 
* fix conflicts

* update bart max length test

* correct spelling mistakes

* implemented model specific encode function

* fix merge conflicts

* better naming

* save intermediate state -> need to rethink strucuture a bit

* leave tf problem as it is for now

* current version

* add layers.pop

* remove ipdb

* make style

* clean return cut decoding

* remove ipdbs

* Fix restoring layers in the decoders that doesnt exists.

* push good intermediate solution for now

* fix conflicts

* always good to refuse to merge conflicts when rebasing

* fix small bug

* improve function calls

* remove unused file

* add correct scope behavior for t5_generate

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2020-03-19 23:18:23 +01:00
Patrick von Platen cec3cdda15 Fix input ids can be none attn mask (#3345) 
* fix issue 3289

* fix attention mask if input_ids None behavior
 2020-03-19 09:55:17 +01:00
Patrick von Platen 292186a3e7 Adding LM Head to Transfo-XL and first step to fixing problem with Adaptive Embeddings in TransfoXL (#3286) 
* first commit

* work in progress

* make language generation task pass

* update to working version for LM

* delete print

* remove dead code

* make style
 2020-03-18 09:24:27 -04:00
Patrick von Platen efdb46b6e2 add link to blog post (#3326) 2020-03-18 13:24:28 +01:00
Patrick von Platen ddb10c6447 improve doctstring (#3327) 2020-03-18 13:24:09 +01:00
Patrick von Platen e8f44af5bf [generate] do_sample default back to False (#3298) 
* change do_samples back

* None better default as boolean

* adapt do_sample to True in test example

* make style
 2020-03-17 10:52:37 -04:00
Patrick von Platen 4759176313 add camembert for Question answering for examples 2020-03-16 14:42:11 -04:00
Patrick von Platen af471ce5e8 Improved Error message when loading config/model with .from_pretrained() (#3247) 
* better error message

* better error message

* update to model identifier instead of url

* update to model identifier instead of ur
 2020-03-16 09:48:30 +01:00
Patrick von Platen ab756f713c add gpt2-xl for tf 2020-03-13 16:40:35 -04:00
Patrick von Platen 4f75d380a4 make style 2020-03-13 16:35:52 +01:00
Patrick von Platen c2ee3840ae update file to new starting token logic 2020-03-13 16:34:44 +01:00
Patrick von Platen 6a82f774f2 fix typo 2020-03-12 21:10:51 +01:00
Patrick von Platen f1c71da115 fix eos_token_ids in test 2020-03-12 21:00:54 +01:00
Patrick von Platen 6047f46b19 re-add eos token to get good bart results 2020-03-12 20:17:50 +01:00
Patrick von Platen c11160114a small clean-up 2020-03-12 20:02:35 +01:00
Patrick von Platen ac303eae46 fix problem with half 2020-03-11 12:24:30 +01:00
Patrick von Platen bc9d5d917c make all tensors half precision 2020-03-11 12:15:38 +01:00
Patrick von Platen a332cc9f7f finalize generation merge 2020-03-11 11:53:36 +01:00
Patrick von Platen 1ba21f96ca fix bug in tf no_repeat_ngram_size 2020-03-11 11:06:56 +01:00
Patrick von Platen d997ac7810 fix typo 2020-03-11 11:06:56 +01:00
Patrick von Platen 7351a8dbaf re-add scoring filtering 2020-03-11 11:06:56 +01:00
Patrick von Platen 9b8ee8cea0 delete print and make style 2020-03-11 11:06:56 +01:00
Patrick von Platen ca1330f0b2 do not mess with the negative sign 2020-03-11 11:06:56 +01:00
Patrick von Platen 10989715d0 rename variable 2020-03-11 11:06:56 +01:00
Patrick von Platen cf06290565 remove ipdb 2020-03-11 11:06:56 +01:00
Patrick von Platen 374deef48d fixed typo 2020-03-11 11:06:56 +01:00
Patrick von Platen a2c8e516c2 fix torch to tf translation 2020-03-11 11:06:56 +01:00
Patrick von Platen ca2047bc35 refactor variable naming and improve tf generate in line with torch generate 2020-03-11 11:06:56 +01:00
patrickvonplaten 41b437ea3a add draft version of propsoed changes for ROGUE score 2020-03-11 11:06:56 +01:00
patrickvonplaten a5751f7578 fix bug with attention_mask as optional input argument 2020-03-11 11:06:56 +01:00
patrickvonplaten 629aac92ec do not allow do_sample and weird force bos token things 2020-03-11 11:06:56 +01:00
patrickvonplaten d880a5fbde finalized PR 2020-03-11 11:06:56 +01:00
patrickvonplaten 2acfe63964 best current version and make style 2020-03-11 11:06:56 +01:00
patrickvonplaten c62444da39 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen 77e6775065 add current changes 2020-03-11 11:06:56 +01:00
Patrick von Platen 333affcb81 add current changes 2020-03-11 11:06:56 +01:00
Patrick von Platen 421216997b comment out stuff 2020-03-11 11:06:56 +01:00
Patrick von Platen 7a11e925cf work in progress 2020-03-11 11:06:56 +01:00
Patrick von Platen 5b3000d933 renamed min_len to min_length 2020-03-11 11:06:56 +01:00
Patrick von Platen aceb3fbaf4 only do output_past=True for language generation in bart 2020-03-11 11:06:56 +01:00
Patrick von Platen 7cba11fb9b better naming 2020-03-11 11:06:56 +01:00
Patrick von Platen ff648221bd fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen c0d9dd3ba9 refactored code a bit and made more generic 2020-03-11 11:06:56 +01:00
Patrick von Platen d8e2b3c547 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen 31f2437f07 Merge pull request #3191 from patrickvonplaten/add_integration_tests_lm_generate_torch_tf 
Add integration tests lm generate torch tf
 2020-03-10 11:29:17 +01:00
Patrick von Platen 49debe62fd Merge pull request #3190 from patrickvonplaten/fix_repetition_penalty_in_tf_generate 
fix repetition penalty mask in tf
 2020-03-09 16:29:57 +01:00
Patrick von Platen 847d370301 fix typo 2020-03-09 16:18:29 +01:00
Patrick von Platen 9050ffe035 delete w! -> need to be more careful with vim 2020-03-09 15:43:12 +01:00