c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,702 Commits 171 Branches 49 Tags
e344e3d4021421ec0d631d076daf17f8a4e82e69
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick von Platen b0ad069517 [Tokenization] fix edge case for bert tokenization (#3517) 
* fix egde gase for bert tokenization

* add Lysandres comments for improvement

* use new is_pretokenized_flag
 2020-04-07 16:26:31 -04:00
Patrick von Platen 80fa0f7812 [Examples, Benchmark] Improve benchmark utils (#3674) 
* improve and add features to benchmark utils

* update benchmark style

* remove output files
 2020-04-07 16:25:57 -04:00
Patrick von Platen 2ee410560e [Generate, Test] Split generate test function into beam search, no beam search (#3601) 
* split beam search and no beam search test

* fix test

* clean generate tests
 2020-04-06 10:37:05 +02:00
Patrick von Platen 1789c7daf1 fix argument order (#3637) 2020-04-05 12:33:41 +02:00
Patrick von Platen b809d2f073 Fix TF T5 docstring (#3636) 2020-04-05 12:23:09 +02:00
Patrick von Platen f68d22850c delete bogus print statement (#3595) 2020-04-02 21:49:34 +02:00
Patrick von Platen 390c128592 [Encoder-Decoder] Force models outputs to always have batch_size as their first dim (#3536) 
* solve conflicts

* improve comments
 2020-04-02 15:18:33 +02:00
Patrick von Platen ab5d06a094 [T5, examples] replace heavy t5 models with tiny random models (#3556) 
* replace heavy t5 models with tiny random models as was done by sshleifer

* fix isort
 2020-04-02 12:34:05 +02:00
Patrick von Platen a4ee4da18a [T5, TF 2.2] change tf t5 argument naming (#3547) 
* change tf t5 argument naming for TF 2.2

* correct bug in testing
 2020-04-01 22:04:20 +02:00
Patrick von Platen 06dd597552 fix bug in warnings T5 pipelines (#3545) 2020-04-01 21:59:12 +02:00
Patrick von Platen b815edf69f [T5, Testst] Add extensive hard-coded integration tests and make sure PT and TF give equal results (#3550) 
* add some t5 integration tests

* finish summarization and translation integration tests for T5 - results loook good

* add tf test

* fix == vs is bug

* fix tf beam search error and make tf t5 tests pass
 2020-04-01 18:01:33 +02:00
Patrick von Platen b38d552a92 [Generate] Add bad words list argument to the generate function (#3367) 
* add bad words list

* make style

* add bad_words_tokens

* make style

* better naming

* make style

* fix typo
 2020-03-31 18:42:31 +02:00
Patrick von Platen ae6834e028 [Examples] Clean summarization and translation example testing files for T5 and Bart (#3514) 
* fix conflicts

* add model size argument to summarization

* correct wrong import

* fix isort

* correct imports

* other isort make style

* make style
 2020-03-31 17:54:13 +02:00
Patrick von Platen 83d1fbcff6 [Docs] Add usage examples for translation and summarization (#3538) 2020-03-31 09:36:03 -04:00
Patrick von Platen 55bcae7f25 remove useless and confusing lm_labels line (#3531) 2020-03-31 09:32:25 -04:00
Patrick von Platen 42e1e3c67f Update usage doc regarding generate fn (#3504) 2020-03-31 09:31:46 -04:00
Patrick von Platen 57b0fab692 Add better explanation to check docs locally. (#3459) 2020-03-31 09:30:17 -04:00
Patrick von Platen 296252c49e fix lm lables in docstring (#3529) 2020-03-30 14:26:24 +02:00
Patrick von Platen 75ec6c9e3a [T5] make decoder input ids optional for t5 training (#3521) 
* make decoder input ids optional for t5 training

* lm_lables should not be shifted in t5

* add tests

* finish shift right functionality for PT T5

* move shift right to correct class

* cleaner code

* replace -100 values with pad token id

* add assert statement

* remove unnecessary for loop

* make style
 2020-03-30 13:45:26 +02:00
Patrick von Platen 5b44e0a31b [T5] Add training documenation (#3507) 
* Add clear description of how to train T5

* correct docstring in T5

* correct typo

* correct docstring format

* update t5 model docs

* implement collins feedback

* fix typo and add more explanation for sentinal tokens

* delete unnecessary todos
 2020-03-30 13:35:53 +02:00
Patrick von Platen 17dceae7a1 Fix circle ci flaky fail of wmt example (#3485) 
* force bleu

* fix wrong file name

* rename file

* different filenames for each example test

* test files should clean up after themselves

* test files should clean up after themselves

* do not force bleu

* correct typo

* fix isort
 2020-03-27 13:01:28 -04:00
Patrick von Platen 00ea100e96 add summarization and translation to notebook (#3478) 2020-03-27 11:05:37 -04:00
Patrick von Platen f4f4946836 Rename t5-large to t5-base in README.md 2020-03-27 15:57:58 +01:00
Patrick von Platen fa9af2468a Add T5 to docs (#3461) 
* add t5 docs basis

* improve docs

* add t5 docs

* improve t5 docstring

* add t5 tokenizer docstring

* finish docstring

* make style

* add pretrained models

* correct typo

* make examples work

* finalize docs
 2020-03-27 10:57:16 -04:00
Patrick von Platen 5ad2ea06af Add wmt translation example (#3428) 
* add translation example

* make style

* adapt docstring

* add gpu device as input for example

* small renaming

* better README
 2020-03-26 19:07:59 +01:00
Patrick von Platen b4fb94fe6d revert unpin isort commit 2020-03-26 13:19:18 -04:00
Patrick von Platen e703e923ca Add t5 summarization example (#3411) 
* rebase to master

* change tf to pytorch

* change to pytorch

* small fix

* renaming

* add gpu training possibility

* renaming

* improve README

* incoorporate collins feedback

* better Readme

* better README.md
 2020-03-26 18:17:55 +01:00
Patrick von Platen 311970546f rename string in pipeline 2020-03-26 14:59:49 +01:00
Patrick von Platen 022e8fab97 Adds translation pipeline (#3419) 
* fix merge conflicts

* add t5 summarization example

* change parameters for t5 summarization

* make style

* add first code snippet for translation

* only add prefixes

* add prefix patterns

* make style

* renaming

* fix conflicts

* remove unused patterns

* solve conflicts

* fix merge conflicts

* remove translation example

* remove summarization example

* make sure tensors are in numpy for float comparsion

* re-add t5 config

* fix t5 import config typo

* make style

* remove unused numpy statements

* update doctstring

* import translation pipeline
 2020-03-26 13:50:58 +01:00
Patrick von Platen 9c683ef01e Add t5 to pipeline(task='summarization') (#3413) 
* solve conflicts

* move warnings below

* incorporate changes

* add pad_to_max_length to pipelines

* add bug fix for T5 beam search

* add prefix patterns

* make style

* fix conflicts

* adapt pipelines for task specific parameters

* improve docstring

* remove unused patterns
 2020-03-26 11:03:13 +01:00
Patrick von Platen ffa17fe322 Extend config with task specific configs. (#3433) 
* add new default configs

* change prefix default to None
 2020-03-25 21:32:04 +01:00
Patrick von Platen e392ba6938 Add camembert integration tests (#3375) 
* add integration tests for camembert

* use jplu/tf-camembert fro the moment

* make style
 2020-03-24 10:18:37 +01:00
Patrick von Platen 95e00d0808 Clean special token init in modeling_....py (#3264) 
* make style

* fix conflicts
 2020-03-20 21:41:04 +01:00
Patrick von PlatenandMorgan Funtowicz bbf26c4e61 Support T5 Generation (#3228) 
* fix conflicts

* update bart max length test

* correct spelling mistakes

* implemented model specific encode function

* fix merge conflicts

* better naming

* save intermediate state -> need to rethink strucuture a bit

* leave tf problem as it is for now

* current version

* add layers.pop

* remove ipdb

* make style

* clean return cut decoding

* remove ipdbs

* Fix restoring layers in the decoders that doesnt exists.

* push good intermediate solution for now

* fix conflicts

* always good to refuse to merge conflicts when rebasing

* fix small bug

* improve function calls

* remove unused file

* add correct scope behavior for t5_generate

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
 2020-03-19 23:18:23 +01:00
Patrick von Platen cec3cdda15 Fix input ids can be none attn mask (#3345) 
* fix issue 3289

* fix attention mask if input_ids None behavior
 2020-03-19 09:55:17 +01:00
Patrick von Platen 292186a3e7 Adding LM Head to Transfo-XL and first step to fixing problem with Adaptive Embeddings in TransfoXL (#3286) 
* first commit

* work in progress

* make language generation task pass

* update to working version for LM

* delete print

* remove dead code

* make style
 2020-03-18 09:24:27 -04:00
Patrick von Platen efdb46b6e2 add link to blog post (#3326) 2020-03-18 13:24:28 +01:00
Patrick von Platen ddb10c6447 improve doctstring (#3327) 2020-03-18 13:24:09 +01:00
Patrick von Platen e8f44af5bf [generate] do_sample default back to False (#3298) 
* change do_samples back

* None better default as boolean

* adapt do_sample to True in test example

* make style
 2020-03-17 10:52:37 -04:00
Patrick von Platen 4759176313 add camembert for Question answering for examples 2020-03-16 14:42:11 -04:00
Patrick von Platen af471ce5e8 Improved Error message when loading config/model with .from_pretrained() (#3247) 
* better error message

* better error message

* update to model identifier instead of url

* update to model identifier instead of ur
 2020-03-16 09:48:30 +01:00
Patrick von Platen ab756f713c add gpt2-xl for tf 2020-03-13 16:40:35 -04:00
Patrick von Platen 4f75d380a4 make style 2020-03-13 16:35:52 +01:00
Patrick von Platen c2ee3840ae update file to new starting token logic 2020-03-13 16:34:44 +01:00
Patrick von Platen 6a82f774f2 fix typo 2020-03-12 21:10:51 +01:00
Patrick von Platen f1c71da115 fix eos_token_ids in test 2020-03-12 21:00:54 +01:00
Patrick von Platen 6047f46b19 re-add eos token to get good bart results 2020-03-12 20:17:50 +01:00
Patrick von Platen c11160114a small clean-up 2020-03-12 20:02:35 +01:00
Patrick von Platen ac303eae46 fix problem with half 2020-03-11 12:24:30 +01:00
Patrick von Platen bc9d5d917c make all tensors half precision 2020-03-11 12:15:38 +01:00
Patrick von Platen a332cc9f7f finalize generation merge 2020-03-11 11:53:36 +01:00
Patrick von Platen 1ba21f96ca fix bug in tf no_repeat_ngram_size 2020-03-11 11:06:56 +01:00
Patrick von Platen d997ac7810 fix typo 2020-03-11 11:06:56 +01:00
Patrick von Platen 7351a8dbaf re-add scoring filtering 2020-03-11 11:06:56 +01:00
Patrick von Platen 9b8ee8cea0 delete print and make style 2020-03-11 11:06:56 +01:00
Patrick von Platen ca1330f0b2 do not mess with the negative sign 2020-03-11 11:06:56 +01:00
Patrick von Platen 10989715d0 rename variable 2020-03-11 11:06:56 +01:00
Patrick von Platen cf06290565 remove ipdb 2020-03-11 11:06:56 +01:00
Patrick von Platen 374deef48d fixed typo 2020-03-11 11:06:56 +01:00
Patrick von Platen a2c8e516c2 fix torch to tf translation 2020-03-11 11:06:56 +01:00
Patrick von Platen ca2047bc35 refactor variable naming and improve tf generate in line with torch generate 2020-03-11 11:06:56 +01:00
patrickvonplaten 41b437ea3a add draft version of propsoed changes for ROGUE score 2020-03-11 11:06:56 +01:00
patrickvonplaten a5751f7578 fix bug with attention_mask as optional input argument 2020-03-11 11:06:56 +01:00
patrickvonplaten 629aac92ec do not allow do_sample and weird force bos token things 2020-03-11 11:06:56 +01:00
patrickvonplaten d880a5fbde finalized PR 2020-03-11 11:06:56 +01:00
patrickvonplaten 2acfe63964 best current version and make style 2020-03-11 11:06:56 +01:00
patrickvonplaten c62444da39 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen 77e6775065 add current changes 2020-03-11 11:06:56 +01:00
Patrick von Platen 333affcb81 add current changes 2020-03-11 11:06:56 +01:00
Patrick von Platen 421216997b comment out stuff 2020-03-11 11:06:56 +01:00
Patrick von Platen 7a11e925cf work in progress 2020-03-11 11:06:56 +01:00
Patrick von Platen 5b3000d933 renamed min_len to min_length 2020-03-11 11:06:56 +01:00
Patrick von Platen aceb3fbaf4 only do output_past=True for language generation in bart 2020-03-11 11:06:56 +01:00
Patrick von Platen 7cba11fb9b better naming 2020-03-11 11:06:56 +01:00
Patrick von Platen ff648221bd fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen c0d9dd3ba9 refactored code a bit and made more generic 2020-03-11 11:06:56 +01:00
Patrick von Platen d8e2b3c547 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen 31f2437f07 Merge pull request #3191 from patrickvonplaten/add_integration_tests_lm_generate_torch_tf 
Add integration tests lm generate torch tf
 2020-03-10 11:29:17 +01:00
Patrick von Platen 49debe62fd Merge pull request #3190 from patrickvonplaten/fix_repetition_penalty_in_tf_generate 
fix repetition penalty mask in tf
 2020-03-09 16:29:57 +01:00
Patrick von Platen 847d370301 fix typo 2020-03-09 16:18:29 +01:00
Patrick von Platen 9050ffe035 delete w! -> need to be more careful with vim 2020-03-09 15:43:12 +01:00
Patrick von Platen efb619235c add print statement to avoid code quality problem 2020-03-09 15:31:21 +01:00
Patrick von Platen b12541c4dc test ctrl 2020-03-09 13:58:01 +00:00
Patrick von Platen 3e624c64ca fix repetition penalty mask in tf 2020-03-09 14:55:11 +01:00
Patrick von Platen b73dd1a0e4 fix typo in test xlm tf 2020-03-09 11:34:31 +01:00
Patrick von Platen 4620caa864 fix if use lang embeddings in tf xlm 2020-03-09 11:18:54 +01:00
patrickvonplaten fbd02d4693 fixed all tests, still need to check ctrl tf and pt and xlm tf 2020-03-08 21:45:55 +01:00
patrickvonplaten b4a3a64744 fix xlnet & transfotests 2020-03-08 16:25:03 +01:00
patrickvonplaten 66c827656f fix typo in test gpt2 2020-03-08 15:35:08 +01:00
patrickvonplaten 314bdc7c14 fix typo in test 2020-03-08 15:34:20 +01:00
patrickvonplaten 575976144a updated all tests 2020-03-08 15:29:10 +01:00
patrickvonplaten 9362eb4a07 refactored beam search according to torch implementation 2020-03-06 00:46:29 +01:00
Patrick von Platen c8035e11e8 Merge pull request #3149 from patrickvonplaten/fix_renaming_error 
fix missed BartForMaskedLM renaming
 2020-03-06 00:45:08 +01:00
patrickvonplaten 58fc8f97a3 fix renaming problem 2020-03-06 00:35:47 +01:00
Patrick von Platen e33ed12c3b uncomment expression 2020-03-05 13:41:04 +01:00
Patrick von Platen 4220fd52b9 remove ipdb 2020-03-05 13:36:21 +01:00
Patrick von Platen c47394b0c9 refactoring and bug fixing beam search generate 2020-03-05 13:12:50 +01:00
Patrick von Platen 932eab943d include tf gpt2 tests for attn mask and past variable (#3122) 2020-03-04 12:03:46 -05:00
Patrick von Platen 006097f8ad rename variables named 'word' to 'token' in generate fn (#3119) 
* fix conflits

* fixed naming bug

* make style
 2020-03-04 12:01:17 -05:00
Patrick von Platen 7a89a3e493 correct beam search sampling 2020-03-04 17:27:47 +01:00