 alberduris
|
81d6841b4b
|
GPU text generation: mMoved the encoded_prompt to correct device
|
2020-01-06 15:11:12 +01:00 |
|
|
alberduris
|
dd4df80f0b
|
Moved the encoded_prompts to correct device
|
2020-01-06 15:11:12 +01:00 |
|
|
Thomas Wolf
|
9f5f646442
|
Merge pull request #2211 from huggingface/fast-tokenizers
Fast tokenizers
|
2019-12-27 10:24:29 +01:00 |
|
|
Anthony MOI
|
2818e50569
|
Add tests for fast tokenizers
|
2019-12-24 13:29:01 -05:00 |
|
|
Aymeric Augustin
|
e6c0019c80
|
Remove unused variables in tests.
|
2019-12-23 22:38:18 +01:00 |
|
|
Aymeric Augustin
|
1c62e87b34
|
Use built-in open().
On Python 3, `open is io.open`.
|
2019-12-22 18:38:56 +01:00 |
|
|
Aymeric Augustin
|
798b3b3899
|
Remove sys.version_info[0] == 2 or 3.
|
2019-12-22 18:38:42 +01:00 |
|
|
Aymeric Augustin
|
c824d15aa1
|
Remove __future__ imports.
|
2019-12-22 17:47:54 +01:00 |
|
|
Aymeric Augustin
|
00204f2b4c
|
Replace CommonTestCases for tokenizers with a mixin.
This is the same change as for (TF)CommonTestCases for modeling.
|
2019-12-22 15:35:25 +01:00 |
|
|
Aymeric Augustin
|
a3c5883f2c
|
Rename file for consistency.
|
2019-12-22 15:35:25 +01:00 |
|