c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[TPU] Doc, fix xla_spawn.py, only preprocess dataset once (#4223)

Browse Source 
* [TPU] Doc, fix xla_spawn.py, only preprocess dataset once

* Update examples/README.md

* [xla_spawn] Add `_mp_fn` to other Trainer scripts

* [TPU] Fix: eval dataloader was None
This commit is contained in:
Julien Chaumond authored and GitHub committed 2020-05-08 14:10:05 -04:00
1 parent 274d850d34
commit 7b75aa9fa5
10 files changed
+88 -47
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

examples/bertology/run_bertology.py
+1 -1
View File
@@ -404,7 +404,7 @@ def main():
logger.info("Training/evaluation parameters %s", args)
# Prepare dataset for the GLUE task
eval_dataset = GlueDataset(args, tokenizer=tokenizer, evaluate=True, local_rank=args.local_rank)
eval_dataset = GlueDataset(args, tokenizer=tokenizer, evaluate=True)
if args.data_subset > 0:
eval_dataset = Subset(eval_dataset, list(range(min(args.data_subset, len(eval_dataset)))))
eval_sampler = SequentialSampler(eval_dataset) if args.local_rank == -1 else DistributedSampler(eval_dataset)