c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,456 Commits 171 Branches 49 Tags
421216997b84d060399f07def9fb7a0ea83c7c2c
Commit Graph
212 Commits
Author SHA1 Message Date
Patrick von Platen 421216997b comment out stuff 2020-03-11 11:06:56 +01:00
Patrick von Platen 7a11e925cf work in progress 2020-03-11 11:06:56 +01:00
Patrick von Platen aceb3fbaf4 only do output_past=True for language generation in bart 2020-03-11 11:06:56 +01:00
Patrick von Platen 7cba11fb9b better naming 2020-03-11 11:06:56 +01:00
Patrick von Platen ff648221bd fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen c0d9dd3ba9 refactored code a bit and made more generic 2020-03-11 11:06:56 +01:00
Patrick von Platen d8e2b3c547 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen 31f2437f07 Merge pull request #3191 from patrickvonplaten/add_integration_tests_lm_generate_torch_tf 
Add integration tests lm generate torch tf
 2020-03-10 11:29:17 +01:00
Julien Chaumond cbf8f5d32b [model upload] Support for organizations 2020-03-09 17:33:57 -04:00
Lysandre 525b6b1c54 TFQA pipeline marked as slow test 2020-03-09 16:52:30 -04:00
Lysandre Debut 5164ea91a7 Skipping outputs (#3116) 
* Minimal example

* Proposal 2

* Proposal 2 for fast tokenizers

* Typings

* Docs

* Revert "Docs" for easier review

This reverts commit eaf0f97062e809887704a542144c537f769d5223.

* Remove unnecessary assignments

* Tests

* Fix faulty type

* Remove prints

* return_outputs -> model_input_names

* Revert "Revert "Docs" for easier review"

This reverts commit 6fdc69408102bf695797f2dfddbb6350c6b9e722.

* code quality
 2020-03-09 13:48:58 -04:00
Patrick von Platen efb619235c add print statement to avoid code quality problem 2020-03-09 15:31:21 +01:00
Patrick von Platen b12541c4dc test ctrl 2020-03-09 13:58:01 +00:00
Patrick von Platen b73dd1a0e4 fix typo in test xlm tf 2020-03-09 11:34:31 +01:00
Patrick von Platen 4620caa864 fix if use lang embeddings in tf xlm 2020-03-09 11:18:54 +01:00
patrickvonplaten fbd02d4693 fixed all tests, still need to check ctrl tf and pt and xlm tf 2020-03-08 21:45:55 +01:00
patrickvonplaten b4a3a64744 fix xlnet & transfotests 2020-03-08 16:25:03 +01:00
patrickvonplaten 66c827656f fix typo in test gpt2 2020-03-08 15:35:08 +01:00
patrickvonplaten 314bdc7c14 fix typo in test 2020-03-08 15:34:20 +01:00
patrickvonplaten 575976144a updated all tests 2020-03-08 15:29:10 +01:00
Sam Shleifer ed37f9fa4f [Bart] _prepare_decoder_inputs should use large negative (#3158) 2020-03-06 16:06:36 -05:00
Thomas Wolf 3e5da38dae Merge pull request #3132 from huggingface/hf_api_model_list 
[hf_api] Get the public list of all the models on huggingface
 2020-03-06 13:05:52 +01:00
Thomas Wolf 9499a3778e Merge pull request #3103 from gthb/keras-serialization 
Support keras JSON/HDF5 serialization of main layers
 2020-03-06 12:59:13 +01:00
patrickvonplaten 58fc8f97a3 fix renaming problem 2020-03-06 00:35:47 +01:00
Sam Shleifer 857e0a0d3b Rename BartForMaskedLM -> BartForConditionalGeneration (#3114) 
* improved documentation
 2020-03-05 17:41:18 -05:00
Lysandre Debut 146c521235 Merge branch 'master' into add_models_special_tokens_to_specific_configs 2020-03-05 17:24:42 -05:00
Lysandre Debut b623ddc000 Pass kwargs to configuration (#3147) 
* Pass kwargs to configuration

* Setter

* test
 2020-03-05 17:16:57 -05:00
Lysandre Debut 0001d05686 Correct missing keys + test (#3143) 2020-03-05 17:01:54 -05:00
sshleifer 1360dacaa3 cleanup deltas 2020-03-05 12:57:42 -05:00
sshleifer c36fdc88d4 tests pass 2020-03-05 12:33:08 -05:00
Julien Chaumond f564f93c84 [hf_api] Get the public list of all the models on huggingface 2020-03-04 23:33:09 -05:00
Julien Chaumond ff9e79ba3a make style 2020-03-04 20:18:07 -05:00
Lysandre 07a79db505 Fix failing doc samples 2020-03-04 19:11:31 -05:00
Thomas Wolf bdd3d0c76d Merge pull request #3118 from patrickvonplaten/add_beam_search_to_generation_tf_2_0 
Add beam search to generation tf 2 0
 2020-03-04 23:28:00 +01:00
Patrick von Platen 932eab943d include tf gpt2 tests for attn mask and past variable (#3122) 2020-03-04 12:03:46 -05:00
patrickvonplaten 61fef6e957 added beam_search generation for tf 2.0 2020-03-04 17:27:47 +01:00
Gunnlaugur Thor Briem 96c4990165 fix unused imports and style 2020-03-03 22:57:05 +00:00
Gunnlaugur Thor Briem 470753bcf5 Put @keras_serializable only on layers it works on 
And only run the test on TF*MainLayer classes so marked.
 2020-03-03 22:44:45 +00:00
Gunnlaugur Thor Briem 0c716ede8c Use class decorator instead of superclass 
When supplied by Keras deserialization, the config parameter to initializers
will be a dict. So intercept it and convert to PretrainedConfig object (and
store in instance attribute for get_config to get at it) before passing to the
actual initializer. To accomplish this, and repeat as little code as possible,
use a class decorator on TF*MainLayer classes.
 2020-03-03 22:31:42 +00:00
Sam Shleifer e9e6efdc45 BartForSequenceClassification: fix num_labels, add test (#3110) 2020-03-03 15:54:29 -05:00
Julien Chaumond f631e01d2c [ci] Re-run integration ground truth from fairseq 
Adopted best practice set by @patrickvonplaten of commenting lines run on fairseq, for easy comparison

also see #3020
 2020-03-03 15:31:40 -05:00
Gunnlaugur Thor Briem b8da16f390 Add (failing) tests for Keras save/load 2020-03-03 15:22:34 +00:00
Patrick von Platen 4134100363 Add generate() functionality to TF 2.0 (#3063) 
* add first copy past test to tf 2 generate

* add tf top_k_top_p_filter fn

* add generate function for TF

* add generate function for TF

* implemented generate for all models expect transfoXL

* implemented generate for all models expect transfoXL

* implemented generate for all models expect transfoXL

* make style

* change permission of test file to correct ones

* delete ipdb

* delete ipdb

* fix bug and finish simple gpt2 integration test

* clean test file

* clean test file

* make style

* make style

* make style

* make style

* change import style

* change import style

* make style

* make style

* add decorators

* add decorators

* fix tf ctrl bug dim => axis in TF

* make style

* make style

* refactored test file

* refactored test file

* take out test_torch_tf_conversion if nothing is defined

* take out test_torch_tf_conversion if nothing is defined

* remove useless files

* remove useless files

* fix conflicts

* fix conflicts

* fix conflicts

* fix conflicts

* fix conflicts

* solve conflicts

* solve conflicts

* fix conflicts

* fix conflicts

* merge conflicts

* delete ipdb

* exposed top_k_top_p_filtering fns

* delete weirdly created w! file

* add comment to test tf common modeling

* fix conflicts

* fix conflicts

* make style

* merge conflicts

* make style

* change tf.tensor.shape to shape_list(tensor)
 2020-03-03 09:42:15 -05:00
Julien Chaumond f169957d0c TF GPU CI (#3085) 
* debug env

* Restrict TF GPU memory

* Fixup

* One more test

* rm debug logs

* Fixup
 2020-03-02 15:45:25 -05:00
Lysandre Debut d3eb7d23a4 Pipeline doc (#3055) 
* Pipeline doc initial commit

* pipeline abstraction

* Remove modelcard argument from pipeline

* Task-specific pipelines can be instantiated with no model or tokenizer

* All pipelines doc
 2020-03-02 14:07:10 -05:00
Julien Chaumond 0e56b37e80 rm bogus file 
cc @patrickvonplaten
 2020-03-02 12:27:12 -05:00
Patrick von Platen 2fdc7f6ce8 correct greedy generation when doing beam search (#3078) 
* correct greedy generation when doing beam search

* improve comment
 2020-03-02 12:00:09 -05:00
Patrick von Platen c0135194eb Force pad_token_id to be set before padding for standard tokenizer (#3035) 
* force pad_token_id to be set before padding

* fix tests and forbid padding without having a padding_token_id set
 2020-03-02 10:53:55 -05:00
Sam Shleifer b54ef78d0c Bart-CNN (#3059) 
`generate` code that produces 99% identical summarizations to fairseq on CNN test data, with caching.
 2020-03-02 10:35:53 -05:00
Lysandre Debut 8bcb37bfb8 NER support for Albert in run_ner.py and NerPipeline (#2983) 
* * Added support for Albert when fine-tuning for NER

* Added support for Albert in NER pipeline

* Added command-line options to examples/ner/run_ner.py to better control tokenization

* Added class AlbertForTokenClassification

* Changed output for NerPipeline to use .convert_ids_to_tokens(...) instead of .decode(...) to better reflect tokens

* Added ,

* Now passes style guide enforcement

* Changes from reviews.

* Code now passes style enforcement

* Added test for AlbertForTokenClassification

* Added test for AlbertForTokenClassification
 2020-02-27 10:22:55 -05:00