 thomwolf
|
ab7bd5ef98
|
fixing tokenization and training
|
2019-08-23 17:31:21 +02:00 |
|
|
thomwolf
|
47d6853439
|
adding max_lengths for single sentences and sentences pairs
|
2019-08-23 17:31:11 +02:00 |
|
|
Thomas Wolf
|
90dcd8c05d
|
Merge branch 'master' into generative-finetuning
|
2019-08-22 10:43:30 +02:00 |
|
|
thomwolf
|
e00b4ff1de
|
fix #1017
|
2019-08-21 22:22:17 +02:00 |
|
|
Lysandre
|
2f9397139d
|
Added GPT-2 LARGE to Pre-trained Models documentation
|
2019-08-21 11:29:37 -04:00 |
|
|
Lysandre
|
d6bbcbc4cf
|
Added finetuning example to documentation
|
2019-08-21 11:22:05 -04:00 |
|
|
VictorSanh
|
6f877d9daf
|
Update dev results on GLUE (bert-base-uncased) w/ median on 5 runs
|
2019-08-21 03:43:29 +00:00 |
|
|
Thomas Wolf
|
07681b6b58
|
Merge pull request #1064 from huggingface/gpt-2-large
Adding gpt-2 large (774M parameters) model
|
2019-08-21 03:05:56 +02:00 |
|
|
thomwolf
|
fdc487d8b3
|
Add max length
|
2019-08-21 02:35:01 +02:00 |
|
|
thomwolf
|
aa05dc8935
|
adding gpt-2 large
|
2019-08-21 02:29:34 +02:00 |
|
|
Thomas Wolf
|
e4515faf54
|
Merge pull request #1057 from huggingface/fixes
Add a few of typos corrections, bugs fixes and small improvements
|
2019-08-21 01:54:05 +02:00 |
|
|
Thomas Wolf
|
41789c6c3d
|
Merge pull request #1059 from GuillemGSubies/master
Better use of spacy tokenizer in open ai and xlm tokenizers
|
2019-08-21 01:53:48 +02:00 |
|
|
Thomas Wolf
|
260c86082d
|
Merge pull request #1027 from samvelyan/iterative_split_on_token
Re-implemented tokenize() iteratively in PreTrainedTokenizer.
|
2019-08-21 01:46:03 +02:00 |
|
|
Thomas Wolf
|
d30cbaf5dc
|
Merge branch 'master' into iterative_split_on_token
|
2019-08-21 01:33:02 +02:00 |
|
|
Thomas Wolf
|
9beaa85b07
|
Merge pull request #1055 from qipeng/run_squad_fix
Fix #1015 (tokenizer defaults to use_lower_case=True when loading from trained models)
|
2019-08-21 01:20:46 +02:00 |
|
|
Thomas Wolf
|
e753f249e1
|
Merge pull request #806 from wschin/fix-a-path
Fix a path so that a test can run on Windows
|
2019-08-21 01:14:40 +02:00 |
|
|
Lysandre
|
2d042274ac
|
Sequence special token handling for BERT and RoBERTa
|
2019-08-20 14:15:28 -04:00 |
|
|
Peng Qi
|
3bffd2e8e5
|
more fixes
|
2019-08-20 10:59:28 -07:00 |
|
|
Thomas Wolf
|
c3619f5536
|
Merge pull request #1060 from CrafterKolyan/patch-1
Fix typo. configuratoin -> configuration
|
2019-08-20 17:39:06 +02:00 |
|
|
Thomas Wolf
|
3b56427a1e
|
Merge pull request #1040 from FeiWang96/multi_gpu
Fix bug of multi-gpu training in lm finetuning
|
2019-08-20 17:13:44 +02:00 |
|
|
thomwolf
|
43489756ad
|
adding proxies options for the from_pretrained methods
|
2019-08-20 16:59:11 +02:00 |
|
|
thomwolf
|
a690edab17
|
various fix and clean up on run_lm_finetuning
|
2019-08-20 15:52:12 +02:00 |
|
|
Nikolay Korolev
|
ad6e62cd82
|
Fix typo. configuratoin -> configuration
|
2019-08-20 15:43:06 +03:00 |
|
|
Guillem García Subies
|
388e3251fa
|
Update tokenization_xlm.py
|
2019-08-20 14:19:39 +02:00 |
|
|
Guillem García Subies
|
f5e2ed0fd8
|
Update tokenization_openai.py
|
2019-08-20 14:19:25 +02:00 |
|
|
Guillem García Subies
|
562b998366
|
Update tokenization_openai.py
|
2019-08-20 14:10:19 +02:00 |
|
|
Guillem García Subies
|
bb04446285
|
Update tokenization_openai.py
|
2019-08-20 14:07:40 +02:00 |
|
|
Guillem García Subies
|
bfd75056b0
|
Update tokenization_xlm.py
|
2019-08-20 14:06:17 +02:00 |
|
|
Thomas Wolf
|
933841d903
|
Merge pull request #1056 from Morizeyao/master
Swap of optimizer.step and scheduler.step for lm finetuning examples
|
2019-08-20 12:42:24 +02:00 |
|
|
thomwolf
|
6d0aa73981
|
fix #1034
|
2019-08-20 12:20:21 +02:00 |
|
|
Julien Chaumond
|
b0b9b8091b
|
minor typo
|
2019-08-20 11:33:46 +02:00 |
|
|
thomwolf
|
53c8f700f4
|
fix #808
|
2019-08-20 11:29:26 +02:00 |
|
|
thomwolf
|
901dde0e45
|
fix #1014
|
2019-08-20 11:05:51 +02:00 |
|
|
thomwolf
|
e239a4a20f
|
close #984
|
2019-08-20 11:02:00 +02:00 |
|
|
thomwolf
|
fecaed0ed4
|
add force_download option to from_pretrained methods
|
2019-08-20 10:56:12 +02:00 |
|
|
Duzeyao
|
d86b49ac86
|
swap optimizer.step and scheduler.step
|
2019-08-20 16:46:34 +08:00 |
|
|
Duzeyao
|
45ab8bf60e
|
Revert "Update finetune_on_pregenerated.py"
This reverts commit a1359b970c.
|
2019-08-20 16:40:39 +08:00 |
|
|
Zeyao Du
|
a1359b970c
|
Update finetune_on_pregenerated.py
|
2019-08-20 16:00:07 +08:00 |
|
|
Zeyao Du
|
28f7ca1f80
|
swap optimizer.step and scheduler.step
|
2019-08-20 15:58:42 +08:00 |
|
|
Peng Qi
|
a368b87791
|
Fix #1015
|
2019-08-19 13:07:00 -07:00 |
|
|
Lysandre
|
f94f1c6016
|
Distributed training + tokenizer agnostic mask token
|
2019-08-19 14:58:50 -04:00 |
|
|
Lysandre
|
c589862b78
|
Doc: loading from config alone does not load the model weights
|
2019-08-19 10:17:47 -04:00 |
|
|
Thomas Wolf
|
5a49b793d9
|
Merge pull request #1023 from tuvuumass/patch-1
fix issue #824
|
2019-08-19 15:31:46 +02:00 |
|
|
Chi-Liang Liu
|
40acf6b52a
|
don't save model without training
|
2019-08-18 05:02:25 -04:00 |
|
|
wangfei
|
856a63da4d
|
Fix: save model/model.module
|
2019-08-18 11:03:47 +08:00 |
|
|
wangfei
|
1ef41b8337
|
Revert "Fix: save model/model.module"
This reverts commit 00e9c4cc96.
|
2019-08-18 11:03:12 +08:00 |
|
|
wangfei
|
00e9c4cc96
|
Fix: save model/model.module
|
2019-08-18 11:02:02 +08:00 |
|
|
Christophe Bourguignat
|
189ff9b664
|
Update README after RoBERTa addition
|
2019-08-17 13:18:37 -04:00 |
|
|
Jason Phang
|
d8923270e6
|
Correct truncation for RoBERTa in 2-input GLUE
|
2019-08-16 16:30:38 -04:00 |
|
|
Lysandre
|
5652f54ac2
|
Simplified data generator + better perplexity calculator
GPT-2 now obtains ~20 perplexity on WikiText-2
|
2019-08-16 13:49:56 -04:00 |
|