c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3,045 Commits 171 Branches 49 Tags
632675ea88bf7e57fbfb320b12c31cd8d8f217fe
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond 90b7df444f Upload CLI: on win32, use slashes, not os.sep 2020-01-22 22:41:21 -05:00
Julien Chaumond 119dc50e2a Doc tweak on model sharing 2020-01-22 22:40:38 -05:00
Julien Chaumond 34a3c25a30 Fix for XLMRobertaConfig inherits from RobertaConfig 
hat/tip @stefan-it
 2020-01-22 17:50:24 -05:00
Julien Chaumond 1a8e87be4e Line-by-line text dataset (including padding) 2020-01-21 16:57:38 -05:00
Julien Chaumond b94cf7faac change order 2020-01-21 16:57:38 -05:00
Julien Chaumond 2eaa8b6e56 Easier to not support this, as it could be confusing 
cc @lysandrejik
 2020-01-21 16:57:38 -05:00
Julien Chaumond 801aaa5508 make style 2020-01-21 16:57:38 -05:00
Julien Chaumond 56d4ba8ddb [run_lm_finetuning] Train from scratch 2020-01-21 16:57:38 -05:00
Julien Chaumond 23a2cea8cb Tokenizer.from_pretrained: fetch all possible files remotely 2020-01-16 16:47:19 -05:00
Julien Chaumond 99f9243de5 same here, try to not serialize too much if unneeded 2020-01-16 16:47:19 -05:00
Julien Chaumond 9d8fd2d40e tokenizer.save_pretrained: only save file if non-empty 2020-01-16 16:47:19 -05:00
Julien Chaumond d9fa1bad72 Fix failing torchscript test for xlnet 
model.parameters() order is apparently not stable (only for xlnet, for some reason)
 2020-01-15 20:22:21 -05:00
Julien Chaumond a98b2ca8c0 Style + fixup BertJapaneseTokenizer 2020-01-15 19:05:51 -05:00
Julien Chaumond 83a41d39b3 💄 super 2020-01-15 18:33:50 -05:00
Julien Chaumond cd51893d37 Merge branch 'Rexhaif-patch-1' 2020-01-15 18:25:15 -05:00
Julien Chaumond 248aeaa842 Merge branch 'patch-1' of https://github.com/Rexhaif/transformers into Rexhaif-patch-1 2020-01-15 18:22:01 -05:00
Julien Chaumond eb59e9f705 Graduate sst-2 to a canonical one 2020-01-15 16:28:50 +00:00
Julien Chaumond e184ad13cf Close #2392 2020-01-15 15:43:44 +00:00
Julien Chaumond 715fa638a7 Merge branch 'master' into from_scratch_training 2020-01-14 18:58:21 +00:00
Julien Chaumond 764f836d52 Update test_tokenization_auto.py 2020-01-13 22:50:34 -05:00
Julien Chaumond d5831acb07 Update test_tokenization_auto.py 2020-01-13 22:47:33 -05:00
Julien Chaumond ed6cd597cc Update test_tokenization_auto.py 2020-01-13 22:46:35 -05:00
Julien Chaumond 5cb463a714 Update test_tokenization_auto.py 2020-01-13 22:38:29 -05:00
Julien Chaumond afc24ea5d4 In a parallel setup this could fail 2020-01-13 23:44:08 +00:00
Julien Chaumond 894812c652 Fixup mapping 2020-01-13 23:34:19 +00:00
Julien Chaumond b20f11d4ca 🔫 Python35 2020-01-13 23:20:44 +00:00
Julien Chaumond 0304628590 Map configs to models and tokenizers 2020-01-13 23:11:44 +00:00
Julien Chaumond 1fc855e456 [tests] Safety checks on CONFIG_MAPPING 2020-01-13 21:52:55 +00:00
Julien Chaumond 3c86b6f3c5 Py35 doesn't like inline variable types 2020-01-13 20:44:33 +00:00
Julien Chaumond b803b067bf Config to Model mapping 2020-01-13 20:05:20 +00:00
Julien Chaumond cf8a70bf68 More AutoConfig tests 2020-01-11 03:43:57 +00:00
Julien Chaumond 6bb3edc300 Serialize model_type if exists 2020-01-11 03:18:56 +00:00
Julien Chaumond c6f682c1eb flake 2020-01-11 03:18:31 +00:00
Julien Chaumond 4d1c98c012 AutoConfig + other Auto classes honor model_type 2020-01-11 02:46:17 +00:00
Julien Chaumond 2f32dfd33b Convention: name mixins mixins 2020-01-11 01:24:29 +00:00
Julien Chaumond 055e80cfad rm old ConfigTester 2020-01-10 21:36:18 +00:00
Julien Chaumond fd8423321f keep list sorted 2020-01-10 20:36:46 +00:00
Julien Chaumond 0cd81fb99f [isort] declare more third-parties in case no tf install 2020-01-10 20:35:45 +00:00
Julien Chaumond 84c0aa1868 num_parameters helper 2020-01-10 17:40:02 +00:00
Julien Chaumond f599623a99 PreTrainedTokenizerFast: hotfix _convert_encoding 
cc @n1t0
 2020-01-08 15:46:37 -05:00
Julien Chaumond c301faa92b Distributed or parallel setup 2020-01-06 18:41:08 -05:00
Julien Chaumond 78528742f1 Fix syntax + link to community page 2020-01-05 12:43:39 -05:00
Julien Chaumond 629b22adcf [run_lm_finetuning] mask_tokens: document types 2020-01-01 12:55:10 -05:00
Julien Chaumond 594ca6dead [debug] Debug Heisenbug, the old school way. 2019-12-29 10:07:21 -05:00
Julien Chaumond 0df4e62da0 [http] Tweak http user-agent (#2353) 2019-12-29 10:06:50 -05:00
Julien Chaumond 0d467fd6de Typo 2019-12-27 23:06:48 -05:00
Julien Chaumond d8293e84f3 [cli] upload: max number of files at the same time 2019-12-27 23:02:53 -05:00
Julien Chaumond 4d6c93e923 Kill __main__ 2019-12-27 22:55:22 -05:00
Julien Chaumond 9b2badf3c9 [cli] Update doc 2019-12-27 22:54:29 -05:00
Julien Chaumond f78ebc22ad [cli] Add ability to delete remote object 2019-12-27 22:53:49 -05:00
Julien Chaumond ac1b449cc9 [doc] move distilroberta to more appropriate place 
cc @lysandrejik
 2019-12-21 00:09:01 -05:00
Julien Chaumond 3e52915fa7 [RoBERTa] Embeddings: fix dimensionality bug 2019-12-20 19:01:27 -05:00
Julien Chaumond 15d897ff4a [http] customizable requests user-agent 2019-12-19 18:29:22 -05:00
Julien Chaumond f25e9b6f77 [hf_bucket_url] support for cloudfront urls 2019-12-19 18:28:17 -05:00
Julien Chaumond a5a06a851e [doc] Param name consistency 2019-12-19 16:24:20 -05:00
Julien Chaumond 9a399ead25 Revert incorrect #1778 2019-12-19 15:45:48 -05:00
Julien Chaumond 94c99db34c [FinBERT] fix incorrect url 2019-12-17 20:35:25 -05:00
Julien Chaumond 7ffa817390 [s3] mv files and update links 2019-12-17 20:35:25 -05:00
Julien Chaumond a0d386455b Fix outdated tokenizer doc 2019-12-17 20:07:39 -05:00
Julien Chaumond ea636440d1 [roberta.conversion] Do not hardcode vocab size 
and support for fairseq 0.9+
 2019-12-17 18:12:22 -05:00
Julien Chaumond 55397dfb9b CsvPipelineDataFormat: Fix for single-column 2019-12-17 13:10:51 -05:00
Julien Chaumond 3f5ccb183e [doc] Clarify uploads 
cf https://github.com/huggingface/transformers/commit/855ff0e91d8b3bd75a3b1c1316e2efd814373764#commitcomment-36452545
 2019-12-16 18:20:29 -05:00
Julien Chaumond 855ff0e91d [doc] Model upload and sharing 
ping @lysandrejik @thomwolf

Is this clear enough? Anything we should add?
 2019-12-16 12:42:22 -05:00
Julien Chaumond b6d4284b26 [cli] Uploads: fix + test edge case 2019-12-13 22:44:57 -05:00
Julien Chaumond 866d73ca26 [cli] Upload is now compatible with folders 2019-12-13 16:39:08 -05:00
Julien Chaumond 5d67aa21ae [doc] Replicate doc from #2144 2019-12-12 12:39:41 -05:00
Julien Chaumond 1748fdf657 [doc] Fix rst table 2019-12-11 18:32:27 -05:00
Julien Chaumond 36fc52a3b4 Update links to weights 2019-12-11 18:32:27 -05:00
Julien Chaumond 371c5ddfad Py2 tests for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond 5505cf7014 Run tests on Py2 too, for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond 9cb97c0c0f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond 95854c4a2f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond d2100428d3 Update to new test infra and only run conditionally 2019-12-11 18:32:27 -05:00
Julien Chaumond 4f15e5a267 Add tests. 
Maybe not the best possible place for the tests, lmk.
 2019-12-11 17:41:51 -05:00
Julien Chaumond 18e1f751f1 TF support 2019-12-11 17:07:46 -05:00
Julien Chaumond 31e5b5ff22 Fix tests + first example of doc 2019-12-11 15:22:02 -05:00
Julien Chaumond c999a3e505 Allow from_pretrained to take a remote identifier 2019-12-11 12:29:58 -05:00
Julien Chaumond 2e2f9fed55 rm duplicate imports 2019-12-11 11:11:56 -05:00
Julien Chaumond 1d18930462 Harmonize no_cuda flag with other scripts 2019-12-09 20:37:55 -05:00
Julien Chaumond e4679cddce [cli] Uploads: add progress bar (#2078) 
* [cli] Uploads: add progress bar

see https://github.com/huggingface/transformers/pull/2044#discussion_r354057827 for context

* rename + documentation

* Add auto-referential comment
 2019-12-06 11:56:23 -05:00
Julien Chaumond 3ba417e1a8 [cli] ls: Tabular formatting 2019-12-04 18:40:52 -05:00
Julien Chaumond 96fa9a8a70 Python 2 + Post mime-type to S3 2019-12-04 17:22:50 -05:00
Julien Chaumond e4fbf3e2cc CLI for authenticated file sharing 2019-12-04 00:52:23 -05:00
Julien Chaumond 7edb51f3a5 [pplm] split classif head into its own file 2019-12-03 22:07:25 +00:00
Julien ChaumondandPiero Molino f434bfc623 [pplm] Update S3 links 
Co-Authored-By: Piero Molino <w4nderlust@gmail.com>
 2019-12-03 10:53:02 -05:00
Julien Chaumond 3b48806f75 [pplm] README: add setup + tweaks 2019-12-03 10:14:02 -05:00
Julien ChaumondandRosanne Liu 0cb2c90890 readme 
Co-Authored-By: Rosanne Liu <mimosavvy@gmail.com>
 2019-12-03 10:14:02 -05:00
Julien Chaumond 1efb2ae7fc [pplm] move scripts under examples/pplm/ 2019-12-03 10:14:02 -05:00
Julien Chaumond d5faa74cd6 tokenizer white space: revert to previous behavior 2019-12-03 10:14:02 -05:00
Julien Chaumond 0b77d66a6d rm extraneous import 2019-12-03 10:14:02 -05:00
Julien ChaumondpieroRosanne Liu
 		572c24cfa2 PPLM (squashed) 
Co-authored-by: piero <piero@uber.com>
Co-authored-by: Rosanne Liu <mimosavvy@gmail.com>
 2019-12-03 10:14:02 -05:00
Julien Chaumond b5d884d25c Uniformize #1952 2019-11-27 11:05:55 -05:00
Julien Chaumond 8742baa531 Improve test protocol for inputs_embeds in TF 2019-11-26 14:39:47 -05:00
Julien Chaumond cf62bdc962 Improve test protocol for inputs_embeds in TF 
cc @lysandrejik
 2019-11-26 14:37:32 -05:00
Julien Chaumond bdfe21ab24 Change param order for consistency 2019-11-26 13:08:12 -05:00
Julien Chaumond afaa335851 [doc] Fix assets urls 2019-11-23 11:34:45 -05:00
Julien Chaumond 176cd1ce1b [doc] homogenize instructions slightly 2019-11-23 11:18:54 -05:00
Julien Chaumond 3916b334a8 [camembert] Acknowledge the full author list 2019-11-18 09:29:11 -05:00
Julien Chaumond 0477b307c7 [camembert] tokenizer: use additional_special_tokens 2019-11-16 00:11:07 -05:00
Julien Chaumond f9abf73e31 [camembert] realign w/ recent changes 2019-11-16 00:11:07 -05:00