VictorSanh
4d6dfbd376
update extract
2019-10-03 10:27:11 -04:00
VictorSanh
23edebc079
update extract_distilbert
2019-10-03 10:27:11 -04:00
VictorSanh
cbfcfce205
update token_counts
2019-10-03 10:27:11 -04:00
VictorSanh
19e4ebbe3f
grouped_batch_sampler
2019-10-03 10:27:11 -04:00
VictorSanh
594202a934
lm_seqs_dataset
2019-10-03 10:27:11 -04:00
VictorSanh
38084507c4
add distillation_configs
2019-10-03 10:27:11 -04:00
Brian Ma
2195c0d5f9
Evaluation result.txt path changing #1286
2019-10-03 12:49:12 +08:00
Thomas Wolf
963529e29b
Merge pull request #1288 from echan00/master
...
Typo with LM Fine tuning script
2019-10-01 18:46:07 -04:00
thomwolf
f7978f70ec
use format instead of f-strings
2019-10-01 18:45:38 -04:00
Julien Chaumond and Lysandre Debut
b350662955
overflowing_tokens do not really make sense here, let's just return a number
...
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr >
2019-09-30 16:37:09 -04:00
Julien Chaumond
f5bcde0b2f
[multiple-choice] Simplify and use tokenizer.encode_plus
2019-09-30 16:04:55 -04:00
Denny
9478590630
Update run_lm_finetuning.py
...
The previous method, just as phrased, did not exist in the class.
2019-09-27 15:18:42 -03:00
Thomas Wolf
d83d295763
Merge pull request #1337 from mgrankin/fastdataset
...
faster dataset building
2019-09-27 10:35:12 +02:00
thomwolf
da2e47ad15
clean up a little run_tf_glue
2019-09-27 09:41:15 +02:00
thomwolf
528c288fa9
clean up run_tf_glue
2019-09-27 09:40:29 +02:00
VictorSanh
702f589848
fix input in run_glue for distilbert
2019-09-27 00:20:14 -04:00
mgrankin
f71a4577b8
faster dataset building
2019-09-26 16:53:13 +03:00
thomwolf
481d9c4fb5
Merge branch 'master' into tf2
2019-09-26 12:02:54 +02:00
thomwolf
31c23bd5ee
[BIG] pytorch-transformers => transformers
2019-09-26 10:15:53 +02:00
thomwolf
5705333441
add initialization for everybody
2019-09-26 10:06:20 +02:00
thomwolf
7c9f8f93f9
fix tests
2019-09-26 01:59:53 +02:00
thomwolf
d6dde438ea
add batch dimension in encode
2019-09-26 01:45:55 +02:00
thomwolf
4a21c4d88d
add warning if neither pt nor tf are found
2019-09-26 01:30:06 +02:00
thomwolf
3b7fb48c3b
fix loading from tf/pt
2019-09-25 17:46:16 +02:00
thomwolf
a049c8043b
push fix to training
2019-09-25 17:33:16 +02:00
thomwolf
5def3302f4
update run_glue
2019-09-25 12:38:08 +02:00
thomwolf
f71758f7a4
update internal glue processors
2019-09-25 12:00:50 +02:00
thomwolf
b5ec526f85
updated data processor and metrics
2019-09-24 17:10:50 +02:00
LysandreJik
f09e5ecef0
[Proposal] GLUE processors included in library
2019-09-24 09:47:34 -04:00
LysandreJik
c832f43a4d
output_token_type -> token_type_ids
2019-09-24 07:21:38 -04:00
LysandreJik
3927d7756c
Updated the GLUE pre-processing method
2019-09-24 07:15:11 -04:00
LysandreJik
9d44236f70
Updated DistilBERT
2019-09-24 07:03:24 -04:00
Lorenzo Ampil
4b543c3007
Add option to use a 'stop token' which will be used to truncate the output text to everything till right before the 'stop token'
2019-09-22 21:38:38 +08:00
VictorSanh
9f995b99d4
minor fixes
2019-09-19 21:36:06 +00:00
VictorSanh
3fe5c8e8a8
update bert-base-uncased rslts
2019-09-19 19:34:22 +00:00
VictorSanh
354944e607
[distillation] big update w/ new weights
2019-09-19 19:25:21 +00:00
LysandreJik
60414f31a9
GLUE updated with new methods
2019-09-19 10:55:06 +02:00
LysandreJik
bf503158c5
Sentence -> Sequence. Removed output_mask from the special token addition methods.
2019-09-19 10:55:06 +02:00
LysandreJik
de8e14b6c0
Added DistilBERT to run_squad script
2019-09-19 10:55:06 +02:00
LysandreJik
88368c2a16
Added DistilBERT to run_lm_finetuning
2019-09-19 10:55:06 +02:00
LysandreJik
75635072e1
Updated GLUE script to add DistilBERT. Cleaned up unused args in the utils file.
2019-09-19 10:55:06 +02:00
LysandreJik
59057abe52
typo
2019-09-19 10:55:06 +02:00
LysandreJik
bac332fec0
Updated the GLUE data processor. Corrections to RoBERTa and XLNet.
2019-09-19 10:55:06 +02:00
Erik Chan
f0340eccf9
Typo
...
Typo
2019-09-18 13:42:11 -07:00
erenup
8960988f35
fixed to find best dev acc
2019-09-19 01:10:05 +08:00
erenup
46ffc28329
Merge branch 'master' into run_multiple_choice_merge
...
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
2019-09-18 21:43:46 +08:00
erenup
15143fbad6
move run_multiple_choice.py and utils_multiple_choice.py to examples
2019-09-18 21:18:46 +08:00
erenup
3cd6289758
Merge remote-tracking branch 'huggingface/master' into run_multiple_choice_merge
...
# Conflicts:
# examples/contrib/run_swag.py
2019-09-18 21:16:59 +08:00
erenup
36362cf086
move schedule.step after optimizer.step
2019-09-18 21:13:40 +08:00
thomwolf
e768f2322a
update run_openai_gpt to fix #1264
2019-09-18 10:07:47 +02:00
VictorSanh