c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
203 Commits 171 Branches 49 Tags
757750d6f6a5883f40815a8f80ff77eb8b0e82df
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
thomwolf 757750d6f6 fix tests 2018-11-17 11:58:14 +01:00
thomwolf a99b971738 bump up version minor 2018-11-17 10:43:39 +01:00
thomwolf 4e46affc34 updating examples 2018-11-17 10:30:54 +01:00
thomwolf d0673c7dbd fix links 2018-11-17 08:59:29 +01:00
thomwolf 68b937aa40 sub section overviews 2018-11-17 08:55:56 +01:00
thomwolf c54d8b1847 fixing links in readme 2018-11-17 08:46:17 +01:00
thomwolf f920eff8c3 update readme 2018-11-17 08:42:45 +01:00
thomwolf 886cb49792 updating readme and notebooks 2018-11-16 14:31:15 +01:00
thomwolf fd647e8c87 comparison masked LM ok 2018-11-16 11:04:31 +01:00
thomwolf cba85a67b9 fix nan in optimizer_on_cpu 2018-11-15 21:47:41 +01:00
thomwolf 1de35b624b preparing for first release 2018-11-15 20:56:10 +01:00
thomwolf d940eeda54 typo 2018-11-12 15:26:46 +01:00
thomwolf 1cf0a16c67 cleaning up readme 2018-11-12 15:24:47 +01:00
thomwolf 66b0090877 add fp16 training 2018-11-12 15:15:02 +01:00
thomwolf a81a1ef8e9 fixing learning rate schedule when using gradient_accumulation_steps 2018-11-10 16:11:14 +01:00
thomwolf ea85cca8ab adding optimize_on_cpu explanation in readme 2018-11-09 11:42:37 +01:00
thomwolf 5f04aa00ed option to perform optimization and keep the optimizer averages on CPU 2018-11-09 11:28:14 +01:00
thomwolf 9e95cd8cd6 clean up optimizer from unused functions 2018-11-09 11:23:55 +01:00
thomwolf 34a1a01091 update code comment 2018-11-09 09:31:20 +01:00
thomwolf 34bdc8b54f remove duplicate accumulate gradient step arguments 2018-11-09 09:19:45 +01:00
thomwolf 2c5d993ba4 update readme - fix SQuAD model on multi-GPU 2018-11-08 21:22:22 +01:00
thomwolf 48d4a5317c typo fix in output tuple 2018-11-07 23:51:12 +01:00
thomwolf dbc318a4c6 cleaning up - speeding up a bit multi-gpu 2018-11-07 22:22:55 +01:00
thomwolf 6bb7510a50 fixing pre-processing bug - averaging loss for gradient accumulation - no_grad on evaluation 2018-11-07 22:12:41 +01:00
thomwolf a1126237a9 clean up logits extraction logic 2018-11-06 17:31:15 +01:00
thomwolf 2a97fe220b fixing weights initialization in the model and out of span clamping 2018-11-06 17:26:33 +01:00
thomwolf 907d3569c1 cleaning up SQuAD notebook - more explanation - fixing error 2018-11-06 11:13:43 +01:00
thomwolf 88e793f31a fix typos 2018-11-05 16:14:19 +01:00
thomwolf 3914eed505 update readme 2018-11-05 16:09:27 +01:00
thomwolf bab5d13077 update optimizer documentation 2018-11-05 16:09:21 +01:00
thomwolf 7394eb47a5 update readme 2018-11-05 15:35:44 +01:00
thomwolf e6646751ac update notebooks 2018-11-05 15:02:50 +01:00
thomwolf b705c9eff5 remove small script, moved notebooks to notebook folder 2018-11-05 14:55:08 +01:00
thomwolf 3a301d443b update gitignore 2018-11-05 14:53:43 +01:00
thomwolf 711d3f9f2b remove tensorflow_code 2018-11-05 14:53:03 +01:00
thomwolf 7875b1a8e0 notebook update 2018-11-05 14:50:44 +01:00
thomwolf c3527cfbc4 ignore SQuAD targets outside of seq_length 2018-11-05 14:18:48 +01:00
thomwolf 1b99cdf71b script that use a small portion of squad only 2018-11-05 13:54:54 +01:00
thomwolf 2f4765d3ed fix multi-gpu squad loss 2018-11-05 13:46:14 +01:00
thomwolf 955cee33a5 updating SQuAD comparison 2018-11-05 13:21:53 +01:00
thomwolf 5622d8320f allowing to load small number of examples 2018-11-05 13:21:24 +01:00
thomwolf a725db4f6c fixing BertForQuestionAnswering loss computation 2018-11-05 13:21:11 +01:00
thomwolf bb5ce67a14 adding back tf code + adding models comparison on SQuAD 2018-11-05 12:11:32 +01:00
thomwolf 3d291dea4a clean up tests 2018-11-04 21:27:19 +01:00
thomwolf 87da161c2a finishing model test 2018-11-04 21:27:10 +01:00
thomwolf d69b0b0e90 fixes + clean up + mask is long 2018-11-04 21:26:54 +01:00
thomwolf 3ddff783c1 clean up + mask is long 2018-11-04 21:26:44 +01:00
thomwolf 88c1037991 update requirements 2018-11-04 21:26:18 +01:00
thomwolf d0cb9fa2a7 clean up model 2018-11-04 21:26:11 +01:00
thomwolf 6cc651778a update readme 2018-11-04 21:26:03 +01:00
thomwolf efb44a8310 distributed in extract features 2018-11-04 21:25:55 +01:00
thomwolf d9d7d1a462 update float() 2018-11-04 21:25:36 +01:00
thomwolf c6207d85b6 remove old methods 2018-11-04 15:34:00 +01:00
thomwolf 965b2565a0 add distributed training 2018-11-04 15:32:04 +01:00
thomwolf 1ceac85e23 add gradient accumulation 2018-11-04 15:26:14 +01:00
thomwolf 6b0da96b4b clean up 2018-11-04 15:17:55 +01:00
thomwolf 834b485b2e logging + update copyright 2018-11-04 12:07:38 +01:00
thomwolf 1701291ef9 multi-gpu cleanup 2018-11-04 11:54:57 +01:00
thomwolf 5ee171689c what's in loss again 2018-11-04 11:45:44 +01:00
thomwolf 0b7a20c651 add tqdm, clean up logging 2018-11-04 11:07:34 +01:00
thomwolf d4e3cf3520 add numpy import 2018-11-04 10:54:16 +01:00
thomwolf cf366417d5 remove run_squad_pytorch 2018-11-04 09:56:00 +01:00
thomwolf 26bdef4321 fixing verbose_argument 2018-11-04 09:53:29 +01:00
thomwolf d6418c5ef3 tweaking the readme 2018-11-03 23:52:35 +01:00
thomwolf 3b70b270e0 update readme 2018-11-03 23:39:55 +01:00
thomwolf eaa6db92f1 Merge branch 'master' of https://github.com/huggingface/pytorch-pretrained-BERT 2018-11-03 23:35:16 +01:00
thomwolf f8276008df update readme, file names, removing TF code, moving tests 2018-11-03 23:35:14 +01:00
thomwolf 139873f6e3 Merge branch 'master' of https://github.com/huggingface/pytorch-pretrained-BERT 2018-11-03 19:06:17 +01:00
thomwolf 04287a4d68 special edition script 2018-11-03 19:06:15 +01:00
thomwolf 25f73add07 update optimizer run_squad 2018-11-03 17:56:34 +01:00
thomwolf f514cbbf30 update run_squad with tqdm 2018-11-03 17:52:44 +01:00
thomwolf cb76c1ddd3 add model.zero_grad() 2018-11-03 17:40:12 +01:00
thomwolf a4086c5de5 Merge branch 'master' of https://github.com/huggingface/pytorch-pretrained-BERT 2018-11-03 17:38:17 +01:00
thomwolf 088ad45888 fixing optimization 2018-11-03 17:38:15 +01:00
thomwolf 852e4b3c00 Merge branch 'master' of https://github.com/huggingface/pytorch-pretrained-BERT 2018-11-03 12:23:04 +01:00
thomwolf 0d8d2285ba fix optimization_test 2018-11-03 12:23:00 +01:00
thomwolf 45efc9d807 removing f-string 2018-11-03 11:46:18 +01:00
thomwolf 4df6027453 clean up 2018-11-03 11:33:10 +01:00
thomwolf e6d106a01b comment on gelu function 2018-11-03 10:47:10 +01:00
thomwolf 01b1a05343 Merge branch 'master' of https://github.com/huggingface/pytorch-pretrained-BERT 2018-11-03 03:11:15 +01:00
thomwolf 8aa22af0c5 fixing model 2018-11-03 03:11:13 +01:00
thomwolf c84315ec35 model fixes + ipnb fixes 2018-11-02 15:11:16 +01:00
thomwolf c9690e57f8 adding jupyter, updating extract features adding simple test file 2018-11-02 14:25:21 +01:00
thomwolf ebfffa0ab2 updated extract_features 2018-11-02 04:32:18 +01:00
thomwolf 9af479b3b9 conversion run_squad ok 2018-11-02 04:12:20 +01:00
thomwolf 8e81e5e6ff working on squad 2018-11-02 04:07:52 +01:00
thomwolf e61db0d1c0 run_squad WIP 2018-11-02 03:56:14 +01:00
thomwolf c0065af6cb implemented BertForQuestionAnswering 2018-11-02 03:04:34 +01:00
thomwolf 5383fca458 update name 2018-11-02 01:56:25 +01:00
thomwolf dee09a40b2 various fixes 2018-11-02 01:52:54 +01:00
thomwolf 2c731fd129 small tweaks 2018-11-02 01:38:22 +01:00
thomwolf 9343a2311b model training loop working – still have to check that everything is exactly same 2018-11-02 01:31:31 +01:00
thomwolf f690f0e167 run_classifier WIP + added classifier head and initialization to the model 2018-11-02 00:27:50 +01:00
thomwolf 4a0b59e980 run_classifier WIP 2018-11-01 21:05:04 +01:00
thomwolf e1bb7904d8 optimization_pytorch from OpenAI with adapted HP - check if there is weights decay on bias 2018-11-01 20:11:14 +01:00
thomwolf 960ef4df3b probably ok weights convertion script 2018-11-01 19:12:31 +01:00
thomwolf ab0e8932a8 convertion script WIP 2018-11-01 18:00:20 +01:00
thomwolf c5d532e5f6 added conversion script 2018-11-01 17:40:05 +01:00
thomwolf 71557b165e working on model 2018-11-01 14:48:15 +01:00
thomwolf 1b95daa0c1 model conversion WIP 2018-11-01 02:22:50 +01:00