c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,949 Commits 171 Branches 49 Tags
af1ee9e648e644aae45c5250a46d4c25a4e0d04d
Commit Graph
2949 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Anthony MOI 7ead04ce14 FastPreTrainedTokenizer => PreTrainedTokenizerFast 2019-12-26 14:39:39 -05:00
Anthony MOI 1f82a5d910 Update for changes in tokenizers API 2019-12-26 14:37:55 -05:00
Thomas Wolf 8c67b529f6 Merge pull request #2324 from kashif/patch-1 
Typo in serving.py
 2019-12-26 12:38:06 +01:00
Kashif Rasul 7211541ade Typo in serving.py 2019-12-26 12:21:40 +01:00
patrickvonplaten 0f6017bee3 improve comments for examples 2019-12-26 00:35:11 +01:00
patrickvonplaten 87c8fca9bc add example for ctrl text generation in docs 2019-12-26 00:29:19 +01:00
patrickvonplaten 88def24c45 merge conflicts - renamed to previous_token singular 2019-12-26 00:27:16 +01:00
patrickvonplaten 822f725a07 duplicated line for repeating_words_penalty_for_language_generation 2019-12-26 00:25:29 +01:00
patrickvonplaten fc84bd5254 adapt style to predefined style layout 2019-12-25 23:32:44 +01:00
patrickvonplaten deff792bb6 add prepare inputs for transfo_xl and xlnet 2019-12-25 23:17:24 +01:00
patrickvonplaten 9398058e19 add easy tensor shape match test 2019-12-25 23:17:24 +01:00
patrickvonplaten 90cda45e9e add past re-ordering for beam search 2019-12-25 23:17:24 +01:00
patrickvonplaten 6bca56fdb0 check for self.config.mem_len instead of self.mem_len in _do_output_past 2019-12-25 23:17:24 +01:00
patrickvonplaten 365ccd0af2 make if statements cleaner for prepare_inputs_for_generation 2019-12-25 23:17:24 +01:00
patrickvonplaten d039c679d2 better naming for if statement 2019-12-25 23:17:24 +01:00
patrickvonplaten 7e0c5c731a changed do_output_past function to check for self.config.output_past instead of self.output_past 2019-12-25 23:17:24 +01:00
patrickvonplaten eeaa402cd4 rename comments 2019-12-25 23:17:24 +01:00
patrickvonplaten 7bb4271291 remove ipdb debugging statements 2019-12-25 23:17:24 +01:00
patrickvonplaten 267587c258 add and improve comments 2019-12-25 23:17:24 +01:00
patrickvonplaten d891fd0ae0 add past hidden key states for more efficient language generation & add prepare_inputs for gpt2 and ctrl model 2019-12-25 23:17:24 +01:00
Thomas Wolf aeef4823ab Merge pull request #2303 from patrickvonplaten/fix_error_with_repetition_penalty 
fix repetition penalty error in modeling_utils.py
 2019-12-25 22:39:20 +01:00
Thomas Wolf 0412f3d929 Merge pull request #2291 from aaugustin/fix-flake8-F841 
Fix F841 flake8 warning
 2019-12-25 22:37:42 +01:00
Thomas Wolf 8742c95461 Merge pull request #2289 from patrickvonplaten/fix_effective_batch_size_lang_gen_xlm 
fix bug in prepare inputs for language generation for xlm for effective batch_size > 1
 2019-12-25 22:30:46 +01:00
Thomas Wolf 1240be3ed9 Merge pull request #2312 from vitaliyradchenko/fix_special_and_add_tokens_loading 
Correct tokenization for special and added tokens
 2019-12-25 20:52:30 +01:00
vitaliyradchenko b262577d17 add special tokens to unique_added_tokens_encoder 2019-12-25 18:31:35 +02:00
vitaliyradchenko 83a2347952 fixed lack of added and special tokens 2019-12-25 18:03:19 +02:00
Thomas Wolf cea04a2443 Merge pull request #2310 from ShnitzelKiller/scatter-unfix 
revert erroneous fix #2276
 2019-12-25 12:43:22 +01:00
James Noeckel e1844d9a45 use positional arguments due to inconsistent API 2019-12-25 01:34:02 -08:00
James Noeckel 9fb7addd4d revert erroneous fix 2019-12-24 22:26:09 -08:00
Anthony MOI 734d29b03d tokenizers is now a real dependency 2019-12-24 13:32:41 -05:00
Anthony MOI 2818e50569 Add tests for fast tokenizers 2019-12-24 13:29:01 -05:00
Anthony MOI 31c56f2e0b Fix style 2019-12-24 12:43:27 -05:00
Anthony MOI 951ae99bea BertTokenizerFast 2019-12-24 12:24:24 -05:00
Anthony MOI 041eac2d6d GPT2TokenizerFast 2019-12-24 12:24:14 -05:00
Anthony MOI 3471ff0d35 FastPreTrainedTokenizer 2019-12-24 12:23:30 -05:00
patrickvonplaten 18e5bdbec5 fix repetition penalty error in modeling_utils.py 2019-12-24 17:18:05 +01:00
patrickvonplaten f18ac4c28e fix sequence length for prepare_inputs for xlnet 2019-12-24 16:43:24 +01:00
patrickvonplaten 359dc43837 fix effective batch_size error in prepare_inputs also for xlnet 2019-12-24 16:33:20 +01:00
patrickvonplaten d98a384cb0 fix bug in prepare inputs for language generation for xlm for effective batch_size > 1 2019-12-24 16:29:54 +01:00
thomwolf 3e0cf49514 adding back last dropout in TF 2.0 T5 2019-12-24 11:30:56 +01:00
thomwolf 35d32308de adding back final dropout in T5 2019-12-24 11:29:49 +01:00
Thomas Wolf 81db12c3ba Merge pull request #2271 from aaugustin/improve-setup-and-requirements 
Improve setup and requirements
 2019-12-24 11:21:20 +01:00
Aymeric Augustin 10724a8123 Run the slow tests every Monday morning. 2019-12-24 09:09:43 +01:00
Aymeric Augustin a8d34e534e Remove [--editable] in install instructions. 
Use -e only in docs targeted at contributors.

If a user copy-pastes  command line with [--editable], they will hit
an error. If they don't know the --editable option, we're giving them
a choice to make before they can move forwards, but this isn't a choice
they need to make right now.
 2019-12-24 08:46:08 +01:00
Aymeric Augustin e74c73a85d Enable F841 warning in flake8. 2019-12-23 22:38:23 +01:00
Aymeric Augustin e6c0019c80 Remove unused variables in tests. 2019-12-23 22:38:18 +01:00
Aymeric Augustin 495580dad1 Remove unused variables in templates. 2019-12-23 22:38:18 +01:00
Aymeric Augustin 71f94a8a1c Remove unused variables in src. 2019-12-23 22:38:09 +01:00
Aymeric Augustin 81422c4e6d Remove unused variables in examples. 2019-12-23 22:29:02 +01:00
Aymeric Augustin 072750f4dc Merge pull request #2288 from aaugustin/better-handle-optional-imports 
Improve handling of optional imports
 2019-12-23 22:28:47 +01:00