thomwolf
0227b4a940
fix #827
2019-07-23 14:06:43 +02:00
Anish Moorthy
490ebbdcf7
Fix PretrainedModel.from_pretrained not passing cache_dir forward
2019-07-22 18:03:08 -04:00
Anish Moorthy
b8009cb0da
Make PreTrainedModel.from_pretrained pass unused arguments to model
2019-07-22 18:03:08 -04:00
Yiqing-Zhou
bef0c629ca
fix
...
Remove '\n' before adding token into vocab
2019-07-22 22:30:49 +08:00
Yiqing-Zhou
897d0841be
read().splitlines() -> readlines()
...
splitlines() does not work as what we expect here for bert-base-chinese because there is a '\u2028' (unicode line seperator) token in vocab file. Value of '\u2028'.splitlines() is ['', ''].
Perhaps we should use readlines() instead.
2019-07-22 20:49:09 +08:00
Wei-Sheng Chin
c4e9615691
Fix a path so that test can run on Windows
2019-07-17 09:08:40 -07:00
Minho Ryu
cd8980e1f4
import sys twice
2019-07-17 18:12:01 +09:00
thomwolf
5fe0b378d8
adding missing docstring fix #793
2019-07-16 21:35:53 +02:00
thomwolf
ed7549bb1a
release version 1.0
2019-07-16 16:10:58 +02:00
thomwolf
4acaa65068
model in evaluation mode by default after from_pretrained
2019-07-16 15:41:57 +02:00
thomwolf
f289e6cfe4
fix docstrings
2019-07-16 15:31:21 +02:00
thomwolf
9726b229cf
model name typo
2019-07-16 15:17:45 +02:00
thomwolf
1849aa7d39
update readme and pretrained model weight files
2019-07-16 15:11:29 +02:00
thomwolf
f31154cb9d
Merge branch 'xlnet'
2019-07-16 11:51:13 +02:00
thomwolf
1b35d05d4b
update conversion scripts and __main__
2019-07-16 09:41:55 +02:00
thomwolf
352e3ff998
added migration guide to readme
2019-07-16 09:03:49 +02:00
thomwolf
3b8b0e01bb
update readme
2019-07-16 00:12:55 +02:00
thomwolf
e691fc0963
update QA models tests + run_generation
2019-07-15 17:45:24 +02:00
thomwolf
15d8b1266c
update tokenizer - update squad example for xlnet
2019-07-15 17:30:42 +02:00
thomwolf
3b469cb422
updating squad for compatibility with XLNet
2019-07-15 15:28:37 +02:00
thomwolf
8ca767f13c
clean up optimization
2019-07-15 13:49:07 +02:00
thomwolf
74a24f0fe9
clean up file_utils
2019-07-15 13:49:01 +02:00
thomwolf
ab49fafc04
update tokenization docstrings for #328
2019-07-15 12:51:23 +02:00
thomwolf
a9ab15174c
fix #328
2019-07-15 12:42:12 +02:00
thomwolf
f7cd7392fd
fixed tests
2019-07-15 12:32:19 +02:00
thomwolf
e28d8bde0d
doc on base classes
2019-07-15 12:08:06 +02:00
thomwolf
44c985facd
update doc for XLM and XLNet
2019-07-15 11:36:50 +02:00
thomwolf
0201d86015
added doc for transformer-xl
2019-07-15 10:11:09 +02:00
thomwolf
4cb489457f
added doc for openai GPT
2019-07-15 09:58:01 +02:00
thomwolf
62b8eb43c1
fix add_start_docstrings on python 2 (removed)
2019-07-15 09:49:02 +02:00
thomwolf
5bc3d0cc5b
added gpt2 doc
2019-07-15 09:40:05 +02:00
thomwolf
183fedfed5
fix doc on python2
2019-07-15 09:00:09 +02:00
thomwolf
2397f958f9
updating examples and doc
2019-07-14 23:20:10 +02:00
thomwolf
7d4b200e40
good quality generation example for GPT, GPT-2, Transfo-XL, XLNet
2019-07-13 15:25:03 +02:00
thomwolf
699bc7e86e
fix gpt-2 unk token test
2019-07-12 11:46:57 +02:00
thomwolf
7442956361
save config file
2019-07-12 11:26:16 +02:00
Thomas Wolf
c57e9d946f
Merge branch 'xlnet' into embeddings
2019-07-12 11:10:14 +02:00
thomwolf
2918b7d2a0
updating tests
2019-07-12 10:57:58 +02:00
LysandreJik
3fbceed8d2
Fix layer reference loss + previous attempted fix
2019-07-11 22:29:55 -04:00
LysandreJik
6c2ee16c04
Test suite testing the tie_weights function as well as the resize_token_embeddings function.
...
Patched an issue relating to the tied weights I had introduced with the TorchScript addition.
Byte order mark management in TSV glue reading.
2019-07-11 22:09:16 -04:00
LysandreJik
e3fb4310d6
From pretrained correct initialization. Unknown token handling for gpt2.
2019-07-11 18:44:29 -04:00
thomwolf
bd404735a7
embeddings resizing + tie_weights
2019-07-12 00:02:49 +02:00
LysandreJik
50e62a4cb4
fix gpt/gpt-2 from pretrained
2019-07-11 16:50:21 -04:00
thomwolf
273617b86d
update config - fix gpt/gpt-2 from pretrained
2019-07-11 22:45:03 +02:00
thomwolf
c6bf1a400d
fix test examples et model pretrained
2019-07-11 22:29:08 +02:00
thomwolf
92a782b108
fix run_glue test
2019-07-11 22:20:10 +02:00
thomwolf
ccb6947dc1
optimization tests
2019-07-11 17:39:47 +02:00
Thomas Wolf
b87eb82b4f
Merge branch 'xlnet' into doc-sphinx
2019-07-11 15:46:27 +02:00
thomwolf
b21d84b027
update examples
2019-07-11 15:37:34 +02:00
thomwolf
ec07cf5a66
rewamp optimization
2019-07-11 14:48:22 +02:00
thomwolf