 LysandreJik
|
39f426be65
|
Added special tokens <pad> and <mask> to RoBERTa.
|
2019-08-13 15:19:50 -04:00 |
|
|
LysandreJik
|
3d87991f60
|
Fixed error with encoding
|
2019-08-13 12:00:24 -04:00 |
|
|
LysandreJik
|
634a3172d8
|
Added integration tests for sequence builders.
|
2019-08-12 15:14:15 -04:00 |
|
|
LysandreJik
|
22ac004a7c
|
Added documentation and changed parameters for special_tokens_sentences_pair.
|
2019-08-12 15:13:53 -04:00 |
|
|
Julien Chaumond
|
b3d83d68db
|
Fixup 9d0603148b
|
2019-08-12 12:28:55 -04:00 |
|
|
LysandreJik
|
75d5f98fd2
|
Roberta tokenization + fixed tests (py3 + py2).
|
2019-08-09 15:02:13 -04:00 |
|
|
LysandreJik
|
14e970c271
|
Tokenization encode/decode class-based sequence handling
|
2019-08-09 15:01:38 -04:00 |
|
|
LysandreJik
|
fbd746bd06
|
Updated test architecture
|
2019-08-08 18:21:34 -04:00 |
|
|
LysandreJik
|
6c41a8f5dc
|
Encode and Decode are back in the superclass. They now handle sentence pairs special tokens.
|
2019-08-08 18:20:32 -04:00 |
|
|
Julien Chaumond
|
e367ac469c
|
[RoBERTa] Re-apply 39d72bcc7b
cc @lysandrejik
|
2019-08-08 11:26:11 -04:00 |
|
|
Julien Chaumond
|
9d0603148b
|
[RoBERTa] RobertaForSequenceClassification + conversion
|
2019-08-08 11:24:54 -04:00 |
|
|
LysandreJik
|
d2cc6b101e
|
Merge branch 'master' into RoBERTa
|
2019-08-08 09:42:05 -04:00 |
|
|
LysandreJik
|
39d72bcc7b
|
Fixed the RoBERTa checkpoint conversion script according to the LM head refactoring.
|
2019-08-07 14:21:57 -04:00 |
|
|
LysandreJik
|
770043eea2
|
Sentence-pair tasks handling. Using common tests on RoBERTa. Forced push to fix indentation.
|
2019-08-07 12:53:19 -04:00 |
|
|
Thomas Wolf
|
7729ef7381
|
Merge pull request #955 from FeiWang96/master
Fix comment typo
|
2019-08-07 10:11:25 +02:00 |
|
|
Thomas Wolf
|
5c6ecf37e7
|
Merge pull request #958 from saket404/typo-fix
Fixed small typo
|
2019-08-07 10:10:20 +02:00 |
|
|
Thomas Wolf
|
822d6768eb
|
Merge pull request #962 from guotong1988/patch-1
Update modeling_xlnet.py
|
2019-08-07 10:09:20 +02:00 |
|
|
Thomas Wolf
|
7e6102ce74
|
Merge pull request #963 from guotong1988/patch-2
Update modeling_bert.py
|
2019-08-07 10:09:04 +02:00 |
|
|
wangfei
|
6ec1ee9ec2
|
Fix examples in docstring
|
2019-08-06 11:32:54 +08:00 |
|
|
wangfei
|
72622926e5
|
Fix examples in docstring
|
2019-08-06 11:32:41 +08:00 |
|
|
wangfei
|
f889e77b9c
|
Fix examples of loading pretrained models in docstring
|
2019-08-06 11:30:35 +08:00 |
|
|
wangfei
|
beb03ec6c5
|
Fix examples of loading pretrained models in docstring
|
2019-08-06 11:24:46 +08:00 |
|
|
Thomas Wolf
|
d43dc48b34
|
Merge branch 'master' into auto_models
|
2019-08-05 19:17:35 +02:00 |
|
|
thomwolf
|
0b524b0848
|
remove derived classes for now
|
2019-08-05 19:08:19 +02:00 |
|
|
thomwolf
|
13936a9621
|
update doc and tests
|
2019-08-05 18:48:16 +02:00 |
|
|
thomwolf
|
ed4e542260
|
adding tests
|
2019-08-05 18:14:07 +02:00 |
|
|
thomwolf
|
3a126e73dd
|
fix #950
|
2019-08-05 17:26:29 +02:00 |
|
|
thomwolf
|
b90e29d52c
|
working on automodels
|
2019-08-05 16:06:34 +02:00 |
|
|
thomwolf
|
328afb7097
|
cleaning up tokenizer tests structure (at last) - last remaining ppb refs
|
2019-08-05 14:08:56 +02:00 |
|
|
Thomas Wolf
|
0e918707dc
|
Merge pull request #907 from dhpollack/fix_convert_to_tf
Fix convert to tf
|
2019-08-05 12:55:04 +02:00 |
|
|
Julien Chaumond
|
cb9db101c7
|
Python 2 must DIE
|
2019-08-04 22:04:15 -04:00 |
|
|
Julien Chaumond
|
05c083520a
|
[RoBERTa] model conversion, inference, tests 🔥
|
2019-08-04 21:39:21 -04:00 |
|
|
雷打不动！
|
d7fd10568c
|
Update modeling_bert.py
|
2019-08-05 08:58:19 +08:00 |
|
|
雷打不动！
|
84eb699082
|
Update modeling_xlnet.py
|
2019-08-05 08:57:09 +08:00 |
|
|
thomwolf
|
00132b7a7a
|
updating docs - adding few tests to tokenizers
|
2019-08-04 22:42:55 +02:00 |
|
|
thomwolf
|
009273dbdd
|
big doc update [WIP]
|
2019-08-04 12:14:57 +02:00 |
|
|
Saket Khandelwal
|
836e513698
|
Fixed small typo
|
2019-08-04 16:05:10 +10:00 |
|
|
wangfei
|
a24f830604
|
Fix comment typo
|
2019-08-03 12:17:06 +08:00 |
|
|
David Pollack
|
c90119e543
|
spelling mistake
|
2019-07-29 16:56:02 +02:00 |
|
|
thomwolf
|
bfbe52ec39
|
cleaning up example docstrings
|
2019-07-27 20:25:39 +02:00 |
|
|
thomwolf
|
4cc1bf81ee
|
typos
|
2019-07-27 12:08:21 +02:00 |
|
|
thomwolf
|
ac27548b25
|
fix unk_token test
|
2019-07-27 11:50:47 +02:00 |
|
|
thomwolf
|
c717d38573
|
dictionnary => dictionary
|
2019-07-26 23:30:48 +02:00 |
|
|
thomwolf
|
7b6e474c9a
|
fix #901
|
2019-07-26 21:26:44 +02:00 |
|
|
thomwolf
|
632d711411
|
fix #908
|
2019-07-26 21:14:37 +02:00 |
|
|
thomwolf
|
27b0f86d36
|
clean up pretrained
|
2019-07-26 17:09:21 +02:00 |
|
|
thomwolf
|
57e54ec070
|
add unk_token to gpt2
|
2019-07-26 17:09:07 +02:00 |
|
|
thomwolf
|
ac42049c08
|
add auto models and auto tokenizer
|
2019-07-26 17:08:59 +02:00 |
|
|
David Pollack
|
09ecf225e9
|
fixed the fix. tf session madness.
|
2019-07-26 15:20:44 +02:00 |
|
|
David Pollack
|
edfd965ac8
|
fix convert_to_tf
|
2019-07-26 14:13:46 +02:00 |
|