 LysandreJik
|
bf503158c5
|
Sentence -> Sequence. Removed output_mask from the special token addition methods.
|
2019-09-19 10:55:06 +02:00 |
|
|
thomwolf
|
69da972ace
|
added test and debug tokenizer configuration serialization
|
2019-08-30 17:09:36 +02:00 |
|
|
LysandreJik
|
634a3172d8
|
Added integration tests for sequence builders.
|
2019-08-12 15:14:15 -04:00 |
|
|
thomwolf
|
328afb7097
|
cleaning up tokenizer tests structure (at last) - last remaining ppb refs
|
2019-08-05 14:08:56 +02:00 |
|
|
thomwolf
|
15d8b1266c
|
update tokenizer - update squad example for xlnet
|
2019-07-15 17:30:42 +02:00 |
|
|
thomwolf
|
c079d7ddff
|
fix python 2 tests
|
2019-07-09 10:40:59 +02:00 |
|
|
thomwolf
|
b19786985d
|
unified tokenizer api and serialization + tests
|
2019-07-09 10:25:18 +02:00 |
|
|
thomwolf
|
36bca545ff
|
tokenization abstract class - tests for examples
|
2019-07-05 15:02:59 +02:00 |
|
|
thomwolf
|
0bab55d5d5
|
[BIG] name change
|
2019-07-05 11:55:36 +02:00 |
|