c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
738 Commits 171 Branches 49 Tags
b8e2a9c5840ed15248cbd0fc5f0cbf6ba5bb886a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
thomwolf 34ae5bf838 small clean up in tests 2019-04-17 14:52:12 +02:00
thomwolf 23d4554ec0 is python 2 happy now 2019-04-17 14:48:34 +02:00
thomwolf 265550ec34 relax network connection requirements 2019-04-17 14:22:35 +02:00
thomwolf fa76520240 fix file_utils on python 2 2019-04-17 13:32:22 +02:00
thomwolf bcde2c61cb fix #497 2019-04-17 12:35:38 +02:00
thomwolf 929579f3b5 fix #497 2019-04-17 12:35:08 +02:00
thomwolf 31d387604c adding s3 model tests with --runslow 2019-04-17 11:58:27 +02:00
thomwolf 5afa497cbf fix GPT-2 tokenization to work also on python 3... 2019-04-17 11:04:41 +02:00
thomwolf bc70779bf0 fixed GPT-2 tokenization on python 2 2019-04-17 10:56:15 +02:00
thomwolf bdaba1897c updating GPT tokenization 2019-04-16 17:44:06 +02:00
thomwolf 18a8a15f78 improving GPT2 tokenization and adding tests 2019-04-16 17:00:55 +02:00
thomwolf 3571187ef6 fix saving models in distributed setting examples 2019-04-15 16:43:56 +02:00
thomwolf d616022455 fix openai special tokens loading 2019-04-15 16:07:45 +02:00
thomwolf df5d9c3551 load all models on cpu 2019-04-15 15:43:01 +02:00
thomwolf 2499b0a5fc add ptvsd to run_squad 2019-04-15 15:33:04 +02:00
thomwolf 7816f7921f clean up distributed training logging in run_squad example 2019-04-15 15:27:10 +02:00
thomwolf 1135f2384a clean up logger in examples for distributed case 2019-04-15 15:22:40 +02:00
thomwolf cc43307023 update readme 2019-04-15 15:06:10 +02:00
thomwolf 60ea6c59d2 added best practices for serialization in README and examples 2019-04-15 15:00:33 +02:00
thomwolf 179a2c2ff6 update example to work with new serialization semantic 2019-04-15 14:33:23 +02:00
thomwolf b3c6ee0ac1 tokenization updates 2019-04-15 14:24:52 +02:00
thomwolf 20577d8a7c add configuration serialization to readme 2019-04-15 14:21:41 +02:00
thomwolf 9761aa4845 add to_json_file method to configuration classes 2019-04-15 14:12:08 +02:00
thomwolf b17963d82f update readme 2019-04-15 13:44:30 +02:00
thomwolf e8568a3b17 fixing tests 2019-04-15 12:55:38 +02:00
thomwolf 870b734bfd added tokenizers serialization tests 2019-04-15 12:03:56 +02:00
thomwolf 3e65f255dc add serialization semantics to tokenizers - fix transfo-xl tokenizer 2019-04-15 11:47:25 +02:00
thomwolf fe2756ff41 update double head model 2019-04-15 10:04:05 +02:00
thomwolf b509bf7655 updating loss computation 2019-04-12 12:12:33 +02:00
thomwolf 1d203a34c0 back to simple indexing 2019-04-11 23:51:03 +02:00
thomwolf 074c869bbe fix OpenAIGPTMultipleChoiceHead 2019-04-11 20:53:50 +02:00
thomwolf 724eb45cef add stale bot 2019-04-11 17:12:00 +02:00
thomwolf 4bc4c69af9 finetuning any BERT model - fixes #455 2019-04-11 16:57:59 +02:00
thomwolf a05fad8dce fix typo 2019-04-11 13:16:17 +02:00
thomwolf 4a82f4f856 update special token addition 2019-04-11 13:11:22 +02:00
thomwolf 991b8e65f4 Merge branch 'master' of https://github.com/huggingface/pytorch-pretrained-BERT 2019-04-11 11:43:15 +02:00
thomwolf e99b2014cc fixes #471 2019-04-11 11:43:13 +02:00
thomwolf 19666dcb3b Should fix #438 2019-04-03 11:01:01 +02:00
thomwolf 1d8c232324 Fix #436 2019-04-03 10:51:03 +02:00
thomwolf 846b1fd6f8 Fix #419 2019-04-03 10:50:38 +02:00
thomwolf 34561e61a5 update main readme also 2019-03-27 12:00:04 +01:00
thomwolf 361aff6de5 typos 2019-03-27 11:54:59 +01:00
thomwolf cea8ba1d59 adjusted formating and some wording in the readme 2019-03-27 11:53:44 +01:00
thomwolf a98dfe4ced fixing #377 (empty nbest_predictions.json) 2019-03-14 09:57:06 +01:00
thomwolf e5f2d9122c adding absolute imports to gpt2, openai and transfo-xl 2019-03-14 09:55:01 +01:00
thomwolf 7cc35c3104 fix openai gpt example and updating readme 2019-03-06 11:43:21 +01:00
thomwolf 906b638efa updating readme 2019-03-06 10:24:19 +01:00
thomwolf 994d86609b fixing PYTORCH_PRETRAINED_BERT_CACHE use in examples 2019-03-06 10:21:24 +01:00
thomwolf 2dd8f524f5 removing test for long sequences error following #337 2019-03-06 10:10:41 +01:00
thomwolf 5c85fc3977 fix typo - logger info 2019-03-06 10:05:21 +01:00
thomwolf 0202da0271 remove unnecessary example 2019-02-18 13:51:42 +01:00
thomwolf e0855e8929 forgot to add regex to requirements :( 2019-02-18 11:54:51 +01:00
thomwolf ab7f5d2943 simple 2019-02-18 11:33:54 +01:00
thomwolf b450a7faf2 clean up tokenization - fix python 2 tests 2019-02-18 11:27:18 +01:00
thomwolf d44db1145c update readme 2019-02-18 11:12:09 +01:00
thomwolf 690a0dbf36 fix example - masking 2019-02-18 10:50:30 +01:00
thomwolf fbb248a2e4 examples testing 2019-02-18 01:28:18 +01:00
thomwolf 5ff0c60505 language update 2019-02-18 00:55:47 +01:00
thomwolf 210d407245 updating init 2019-02-18 00:55:39 +01:00
thomwolf b65f07d8c0 adding examples 2019-02-18 00:55:33 +01:00
thomwolf 009ee86a19 fix tests - bump up version 2019-02-17 23:57:23 +01:00
thomwolf ffd623823d adding gpt2 2019-02-17 23:38:51 +01:00
thomwolf 321d70a7a9 bump up to 0.5.1 2019-02-13 10:11:20 +01:00
thomwolf 67376c02e2 update readme for tokenizers 2019-02-13 10:11:11 +01:00
thomwolf c6bea08448 OpenAI GPT Tokenizer can fallback on using BERT BasicTokenizer 2019-02-13 10:11:00 +01:00
thomwolf e7cfc46fc1 fix TransfoXLModel loading 2019-02-13 09:32:46 +01:00
thomwolf 1e71f11dec Release: 0.5.0 2019-02-11 14:16:27 +01:00
thomwolf d38caba169 typo in run_squad 2019-02-11 14:10:27 +01:00
thomwolf af62cc5f20 fix run_squad example 2019-02-11 14:06:32 +01:00
thomwolf eebc8abbe2 clarify and unify model saving logic in examples 2019-02-11 14:04:19 +01:00
thomwolf 81c7e3ec9f fix typo in readme 2019-02-11 13:37:12 +01:00
thomwolf e8fe6b7140 adapting transfo tokenizer to transposed inputs 2019-02-11 13:30:04 +01:00
thomwolf 884ca81d87 transposing the inputs of Transformer-XL to have a unified interface 2019-02-11 13:19:59 +01:00
thomwolf 32fea876bb add distant debugging to run_transfo_xl 2019-02-11 12:53:32 +01:00
thomwolf b31ba23913 cuda on in the examples by default 2019-02-11 12:15:43 +01:00
thomwolf 0a9860daa7 tests pass on python 2 and 3 2019-02-11 10:47:52 +01:00
thomwolf 2071a9b86e fix python 2.7 imports 2019-02-11 10:35:36 +01:00
thomwolf 8197eb9f10 update Circle CI config 2019-02-11 10:22:10 +01:00
thomwolf 525eba68ab update Circle CI 2019-02-11 10:19:25 +01:00
thomwolf b514a60c36 added tests for OpenAI GPT and Transformer-XL tokenizers 2019-02-11 10:17:16 +01:00
thomwolf 9bdcba53fd fix tests 2019-02-09 17:07:12 +01:00
thomwolf f0bf81e141 back compatibility with Path inputs in fle_utils 2019-02-09 17:05:23 +01:00
thomwolf 9f9909ea2f update readme 2019-02-09 16:59:21 +01:00
thomwolf 6cd769957e update transfo xl example 2019-02-09 16:59:17 +01:00
thomwolf 1320e4ec0c mc_token_mask => mc_token_ids 2019-02-09 16:58:53 +01:00
thomwolf f4a07a392c mems not splitted 2019-02-09 16:14:31 +01:00
thomwolf 43b9af0cac mems initialized to None in run_transfo 2019-02-09 16:12:19 +01:00
thomwolf cfcb95417c fix hasattr 2019-02-08 23:08:53 +01:00
thomwolf 0c1a6f9b1d update readme 2019-02-08 22:32:25 +01:00
thomwolf 1756b5e956 fix loading from Transfo-XL LM model 2019-02-08 22:32:17 +01:00
thomwolf dadd0c1b13 updating __main__ 2019-02-08 22:31:57 +01:00
thomwolf 102c6b238c adding file cache to __init__ 2019-02-08 22:31:46 +01:00
thomwolf b80684b23f fixing run openai gpt example 2019-02-08 22:31:32 +01:00
thomwolf 80607874c1 fix layer norm epsilon in OpenAI GPT 2019-02-08 21:49:05 +01:00
thomwolf 7b4b0cf966 logging 2019-02-08 11:16:29 +01:00
thomwolf 4bbb9f2d68 log loss - helpers 2019-02-08 11:14:29 +01:00
thomwolf 5d7e845712 fix model on cuda 2019-02-08 11:08:43 +01:00
thomwolf eccb2f0163 hot fix 2019-02-08 11:05:20 +01:00
thomwolf 5adc20723b add distant debugging 2019-02-08 11:03:59 +01:00
thomwolf 5ee4f17234 adding option to load on cpu 2019-02-08 10:37:40 +01:00