c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,710 Commits 171 Branches 49 Tags
d6dde438eaf7117fa17de0c4ede3e5126989dfdf
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
thomwolf d6dde438ea add batch dimension in encode 2019-09-26 01:45:55 +02:00
thomwolf 4a21c4d88d add warning if neither pt nor tf are found 2019-09-26 01:30:06 +02:00
thomwolf 2967de06f4 adding intialization to bert 2019-09-25 22:08:38 +02:00
thomwolf a6bcfb8015 fix tests 2019-09-25 21:14:12 +02:00
thomwolf 78863f6b36 fix tokenizer to tensors 2019-09-25 21:09:46 +02:00
thomwolf 8a618e0af5 clean up __init__ 2019-09-25 21:04:52 +02:00
thomwolf 3b7fb48c3b fix loading from tf/pt 2019-09-25 17:46:16 +02:00
thomwolf a049c8043b push fix to training 2019-09-25 17:33:16 +02:00
thomwolf 5def3302f4 update run_glue 2019-09-25 12:38:08 +02:00
thomwolf f71758f7a4 update internal glue processors 2019-09-25 12:00:50 +02:00
thomwolf 0f091062d4 Merge branch 'glue-example' into tf2 2019-09-25 10:21:52 +02:00
thomwolf c4acc3a8e9 let encode accept tensor inputs 2019-09-25 10:19:14 +02:00
thomwolf 789ea72037 fix output_token_type in glue 2019-09-24 17:32:01 +02:00
thomwolf 1cbd566c63 Merge branch 'glue-example' into glue-included 2019-09-24 17:24:52 +02:00
thomwolf 743e383d4b py2 fix 2019-09-24 17:21:54 +02:00
thomwolf 99a90e43d4 update data processors __init__ 2019-09-24 17:16:46 +02:00
thomwolf b5ec526f85 updated data processor and metrics 2019-09-24 17:10:50 +02:00
thomwolf a6981076ec various updates 2019-09-24 16:46:26 +02:00
thomwolf 128bdd4c35 fix tests pt/tf 2019-09-24 15:43:39 +02:00
thomwolf 28a30af6d1 fix auto models 2019-09-24 15:33:39 +02:00
thomwolf de203853cc docstring for xlnet 2019-09-24 15:30:55 +02:00
thomwolf 559790f9e4 docstring for xlm 2019-09-24 15:26:57 +02:00
thomwolf b3087ddde8 docstring t-xl 2019-09-24 15:21:51 +02:00
thomwolf 4761a39781 doctring roberta 2019-09-24 15:19:09 +02:00
thomwolf 45a6f2edd9 docstring for GPT 2019-09-24 15:15:47 +02:00
thomwolf e7ba5bc85b docstring for GPT2 2019-09-24 15:12:36 +02:00
thomwolf b94f73bab7 distilbert docstring 2019-09-24 15:06:51 +02:00
thomwolf 9678c49419 docstrings for bert 2019-09-24 14:57:05 +02:00
thomwolf f3d1511b5b fix imports 2019-09-24 14:42:09 +02:00
thomwolf dd2d90f344 update automodels 2019-09-24 14:39:41 +02:00
thomwolf ee261439a9 add save_pretrained 2019-09-24 14:30:28 +02:00
thomwolf 29bb3e4eb0 double loading ok 2019-09-24 14:23:46 +02:00
thomwolf f5397ffc3b update loading logics 2019-09-24 14:03:58 +02:00
thomwolf 271f213621 updating to load tf model in pt - fixing headmasking test 2019-09-24 13:51:28 +02:00
thomwolf cf9c1cbb60 fix tests chen only using tf 2019-09-24 13:32:47 +02:00
thomwolf 2167e366ba update circleCi 2019-09-24 13:27:45 +02:00
thomwolf e9a103c17a bidirectional conversion TF <=> PT - extended tests 2019-09-24 13:25:50 +02:00
thomwolf a7e01a248b converting distilled/fine-tuned models 2019-09-24 10:58:52 +02:00
thomwolf 8ba44ced95 fix roberta conversion script 2019-09-24 09:48:23 +02:00
thomwolf 2b11fa5174 update __init__ and conversion script 2019-09-23 22:35:45 +02:00
thomwolf 6448396d54 fix roberta test 2019-09-23 22:27:13 +02:00
thomwolf 1e47dee24c Merge branch 'tf2' of https://github.com/huggingface/pytorch-transformers into tf2 2019-09-23 22:08:10 +02:00
thomwolf c9591f6fac updated models input format + tests 2019-09-23 22:08:08 +02:00
thomwolf c014d1f0c6 fix the skipping 2019-09-23 16:39:57 +02:00
thomwolf 0b22e47a40 skipping pretrained TF model tests for now 2019-09-23 16:38:03 +02:00
thomwolf 830d212be7 test circleCI h5py version 2019-09-23 16:26:06 +02:00
thomwolf a31e591d27 fix XLM tests 2019-09-23 15:54:10 +02:00
thomwolf 447de34dde tests for distilbert and roberta 2019-09-23 15:38:29 +02:00
thomwolf 68a3e0223a roberta and distilbert 2019-09-20 23:14:51 +02:00
thomwolf 3a527fa820 OpenAI GPT tests ok 2019-09-18 14:15:48 +02:00
thomwolf 556442afb3 hot fix 2019-09-18 14:12:41 +02:00
thomwolf 160b5d6080 fix xlm lang_embeddings loading 2019-09-18 14:10:20 +02:00
thomwolf 26497d1199 fix tests 2019-09-18 12:17:21 +02:00
thomwolf 6a083fd447 update pt-tf conversion script 2019-09-18 12:11:32 +02:00
thomwolf f6969cc12b upgrade max model difference to 2e-2 (for transfo-xl adaptive softmax + inputs) 2019-09-18 11:12:02 +02:00
thomwolf e768f2322a update run_openai_gpt to fix #1264 2019-09-18 10:07:47 +02:00
thomwolf 8334993915 clean up examples - updated to new keyword inputs - #1246 2019-09-18 10:01:27 +02:00
thomwolf 45de034bf8 fix #1223 2019-09-17 10:25:06 +02:00
thomwolf c88f05163d fix typo in XLM models 2019-09-16 13:42:20 +02:00
thomwolf 4b956b2a6b add layer_norm_epsilon configuration for transformer xl 2019-09-13 17:09:20 +02:00
thomwolf b97af8cce9 skip finetuned checkpoints 2019-09-13 16:43:49 +02:00
thomwolf 65c49bb27e adding TF 2.0 adaptive softmax with logits + loss outputs 2019-09-13 15:50:51 +02:00
thomwolf 39c38b2ea0 fix 2019-09-12 16:47:11 +02:00
thomwolf dcddf498c8 fix bert layernorm 2019-09-12 16:46:32 +02:00
thomwolf d3a3a0353c clean up cache after conversion 2019-09-12 16:42:52 +02:00
thomwolf a84adddd1b convert all models 2019-09-12 13:14:07 +02:00
thomwolf 969d3ae95e XLMWithLMHead fixed - standardize conversion 2019-09-11 15:47:33 +02:00
thomwolf 646711e1e2 standardize scopes names - add conversion methods 2019-09-11 15:34:17 +02:00
thomwolf 4356f791a2 XLM passing tests 2019-09-11 11:49:54 +02:00
thomwolf 465870c33f Xlnet working - also added simple question answering model for XLNet 2019-09-10 16:44:41 +02:00
thomwolf 16b6361792 xlnet paassing first test 2019-09-10 12:39:27 +02:00
thomwolf 32aabe8c33 WIP XLNet 2019-09-10 12:17:18 +02:00
thomwolf f851fb55ca fixing error message 2019-09-10 09:24:08 +02:00
thomwolf 50c6bc4195 fix tf bert model 2019-09-09 17:46:01 +02:00
thomwolf 0537139b2b removing tf.function 2019-09-09 14:47:31 +02:00
thomwolf 33cb00f41a add GPT2 to init - fix weights loading - remove tf.function 2019-09-09 14:29:24 +02:00
thomwolf 78b2a53f10 debug file download in tests error 2019-09-09 13:38:10 +02:00
thomwolf 6b3438df21 fixing GPT2 double head model and updating the torch version tests 2019-09-09 12:48:36 +02:00
thomwolf e360037236 Merge branch 'tf2' of https://github.com/huggingface/pytorch-transformers into tf2 2019-09-09 11:08:49 +02:00
thomwolf b7175a2701 fixed imports in tests and gpt2 config test 2019-09-09 11:04:03 +02:00
thomwolf 3401980fc4 fix #1208 2019-09-09 10:22:12 +03:00
thomwolf 728637356c WIP GPT2 2019-09-09 10:18:55 +03:00
thomwolf 34f28b2a13 WIP GPT2 2019-09-08 15:02:06 +03:00
thomwolf ad88563bda WIP GPT-2 2019-09-08 15:02:06 +03:00
thomwolf 64d83c7ae0 WIP 2019-09-08 15:02:06 +03:00
thomwolf 01597e5b90 add tf auto models + tests 2019-09-08 15:02:06 +03:00
thomwolf f5c698b21a add weights tying, attention and hidden states output tests 2019-09-08 15:02:06 +03:00
thomwolf 6dc4b6f34c skip transfo-xl tokenizer tests with tf for now 2019-09-08 15:02:06 +03:00
thomwolf e30579f764 no pytest version checking 2019-09-08 15:02:06 +03:00
thomwolf 518307dfcd test suite independent of framework 2019-09-08 15:02:06 +03:00
thomwolf 9d0a11a68c update dependencies and circle-ci 2019-09-08 15:02:06 +03:00
thomwolf 24a20483f5 update conversion script names 2019-09-08 15:02:06 +03:00
thomwolf 6f152572cd add conversion script, rename conversion scripts 2019-09-08 15:02:06 +03:00
thomwolf a4704b1263 skipping tf tests if tf is not installed 2019-09-08 15:02:06 +03:00
thomwolf ad0ab9afe9 fix test when tf is not here 2019-09-08 15:02:06 +03:00
thomwolf 59fe641b8b also gathering file names in file_utils 2019-09-08 15:02:06 +03:00
thomwolf d68a8fe462 add tf bert files 2019-09-08 15:02:06 +03:00
thomwolf 7ae642b72d update conversion scripts 2019-09-08 15:02:06 +03:00
thomwolf 69bff89935 clean ups 2019-09-08 15:02:06 +03:00
thomwolf 1efb1f1660 split configuration and modeling files 2019-09-08 15:02:06 +03:00