c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,964 Commits 171 Branches 49 Tags
b803b067bfb7740a7b9ae6ae149fbbd24e3f21af
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Julien Chaumond b803b067bf Config to Model mapping 2020-01-13 20:05:20 +00:00
Julien Chaumond cf8a70bf68 More AutoConfig tests 2020-01-11 03:43:57 +00:00
Julien Chaumond 6bb3edc300 Serialize model_type if exists 2020-01-11 03:18:56 +00:00
Julien Chaumond c6f682c1eb flake 2020-01-11 03:18:31 +00:00
Julien Chaumond 4d1c98c012 AutoConfig + other Auto classes honor model_type 2020-01-11 02:46:17 +00:00
Julien Chaumond 2f32dfd33b Convention: name mixins mixins 2020-01-11 01:24:29 +00:00
Julien Chaumond 055e80cfad rm old ConfigTester 2020-01-10 21:36:18 +00:00
Julien Chaumond fd8423321f keep list sorted 2020-01-10 20:36:46 +00:00
Julien Chaumond 0cd81fb99f [isort] declare more third-parties in case no tf install 2020-01-10 20:35:45 +00:00
Julien Chaumond 84c0aa1868 num_parameters helper 2020-01-10 17:40:02 +00:00
Julien Chaumond f599623a99 PreTrainedTokenizerFast: hotfix _convert_encoding 
cc @n1t0
 2020-01-08 15:46:37 -05:00
Julien Chaumond c301faa92b Distributed or parallel setup 2020-01-06 18:41:08 -05:00
Julien Chaumond 78528742f1 Fix syntax + link to community page 2020-01-05 12:43:39 -05:00
Julien Chaumond 629b22adcf [run_lm_finetuning] mask_tokens: document types 2020-01-01 12:55:10 -05:00
Julien Chaumond 594ca6dead [debug] Debug Heisenbug, the old school way. 2019-12-29 10:07:21 -05:00
Julien Chaumond 0df4e62da0 [http] Tweak http user-agent (#2353) 2019-12-29 10:06:50 -05:00
Julien Chaumond 0d467fd6de Typo 2019-12-27 23:06:48 -05:00
Julien Chaumond d8293e84f3 [cli] upload: max number of files at the same time 2019-12-27 23:02:53 -05:00
Julien Chaumond 4d6c93e923 Kill __main__ 2019-12-27 22:55:22 -05:00
Julien Chaumond 9b2badf3c9 [cli] Update doc 2019-12-27 22:54:29 -05:00
Julien Chaumond f78ebc22ad [cli] Add ability to delete remote object 2019-12-27 22:53:49 -05:00
Julien Chaumond ac1b449cc9 [doc] move distilroberta to more appropriate place 
cc @lysandrejik
 2019-12-21 00:09:01 -05:00
Julien Chaumond 3e52915fa7 [RoBERTa] Embeddings: fix dimensionality bug 2019-12-20 19:01:27 -05:00
Julien Chaumond 15d897ff4a [http] customizable requests user-agent 2019-12-19 18:29:22 -05:00
Julien Chaumond f25e9b6f77 [hf_bucket_url] support for cloudfront urls 2019-12-19 18:28:17 -05:00
Julien Chaumond a5a06a851e [doc] Param name consistency 2019-12-19 16:24:20 -05:00
Julien Chaumond 9a399ead25 Revert incorrect #1778 2019-12-19 15:45:48 -05:00
Julien Chaumond 94c99db34c [FinBERT] fix incorrect url 2019-12-17 20:35:25 -05:00
Julien Chaumond 7ffa817390 [s3] mv files and update links 2019-12-17 20:35:25 -05:00
Julien Chaumond a0d386455b Fix outdated tokenizer doc 2019-12-17 20:07:39 -05:00
Julien Chaumond ea636440d1 [roberta.conversion] Do not hardcode vocab size 
and support for fairseq 0.9+
 2019-12-17 18:12:22 -05:00
Julien Chaumond 55397dfb9b CsvPipelineDataFormat: Fix for single-column 2019-12-17 13:10:51 -05:00
Julien Chaumond 3f5ccb183e [doc] Clarify uploads 
cf https://github.com/huggingface/transformers/commit/855ff0e91d8b3bd75a3b1c1316e2efd814373764#commitcomment-36452545
 2019-12-16 18:20:29 -05:00
Julien Chaumond 855ff0e91d [doc] Model upload and sharing 
ping @lysandrejik @thomwolf

Is this clear enough? Anything we should add?
 2019-12-16 12:42:22 -05:00
Julien Chaumond b6d4284b26 [cli] Uploads: fix + test edge case 2019-12-13 22:44:57 -05:00
Julien Chaumond 866d73ca26 [cli] Upload is now compatible with folders 2019-12-13 16:39:08 -05:00
Julien Chaumond 5d67aa21ae [doc] Replicate doc from #2144 2019-12-12 12:39:41 -05:00
Julien Chaumond 1748fdf657 [doc] Fix rst table 2019-12-11 18:32:27 -05:00
Julien Chaumond 36fc52a3b4 Update links to weights 2019-12-11 18:32:27 -05:00
Julien Chaumond 371c5ddfad Py2 tests for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond 5505cf7014 Run tests on Py2 too, for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond 9cb97c0c0f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond 95854c4a2f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond d2100428d3 Update to new test infra and only run conditionally 2019-12-11 18:32:27 -05:00
Julien Chaumond 4f15e5a267 Add tests. 
Maybe not the best possible place for the tests, lmk.
 2019-12-11 17:41:51 -05:00
Julien Chaumond 18e1f751f1 TF support 2019-12-11 17:07:46 -05:00
Julien Chaumond 31e5b5ff22 Fix tests + first example of doc 2019-12-11 15:22:02 -05:00
Julien Chaumond c999a3e505 Allow from_pretrained to take a remote identifier 2019-12-11 12:29:58 -05:00
Julien Chaumond 2e2f9fed55 rm duplicate imports 2019-12-11 11:11:56 -05:00
Julien Chaumond 1d18930462 Harmonize no_cuda flag with other scripts 2019-12-09 20:37:55 -05:00
Julien Chaumond e4679cddce [cli] Uploads: add progress bar (#2078) 
* [cli] Uploads: add progress bar

see https://github.com/huggingface/transformers/pull/2044#discussion_r354057827 for context

* rename + documentation

* Add auto-referential comment
 2019-12-06 11:56:23 -05:00
Julien Chaumond 3ba417e1a8 [cli] ls: Tabular formatting 2019-12-04 18:40:52 -05:00
Julien Chaumond 96fa9a8a70 Python 2 + Post mime-type to S3 2019-12-04 17:22:50 -05:00
Julien Chaumond e4fbf3e2cc CLI for authenticated file sharing 2019-12-04 00:52:23 -05:00
Julien Chaumond 7edb51f3a5 [pplm] split classif head into its own file 2019-12-03 22:07:25 +00:00
Julien ChaumondandPiero Molino f434bfc623 [pplm] Update S3 links 
Co-Authored-By: Piero Molino <w4nderlust@gmail.com>
 2019-12-03 10:53:02 -05:00
Julien Chaumond 3b48806f75 [pplm] README: add setup + tweaks 2019-12-03 10:14:02 -05:00
Julien ChaumondandRosanne Liu 0cb2c90890 readme 
Co-Authored-By: Rosanne Liu <mimosavvy@gmail.com>
 2019-12-03 10:14:02 -05:00
Julien Chaumond 1efb2ae7fc [pplm] move scripts under examples/pplm/ 2019-12-03 10:14:02 -05:00
Julien Chaumond d5faa74cd6 tokenizer white space: revert to previous behavior 2019-12-03 10:14:02 -05:00
Julien Chaumond 0b77d66a6d rm extraneous import 2019-12-03 10:14:02 -05:00
Julien ChaumondpieroRosanne Liu
 		572c24cfa2 PPLM (squashed) 
Co-authored-by: piero <piero@uber.com>
Co-authored-by: Rosanne Liu <mimosavvy@gmail.com>
 2019-12-03 10:14:02 -05:00
Julien Chaumond b5d884d25c Uniformize #1952 2019-11-27 11:05:55 -05:00
Julien Chaumond 8742baa531 Improve test protocol for inputs_embeds in TF 2019-11-26 14:39:47 -05:00
Julien Chaumond cf62bdc962 Improve test protocol for inputs_embeds in TF 
cc @lysandrejik
 2019-11-26 14:37:32 -05:00
Julien Chaumond bdfe21ab24 Change param order for consistency 2019-11-26 13:08:12 -05:00
Julien Chaumond afaa335851 [doc] Fix assets urls 2019-11-23 11:34:45 -05:00
Julien Chaumond 176cd1ce1b [doc] homogenize instructions slightly 2019-11-23 11:18:54 -05:00
Julien Chaumond 3916b334a8 [camembert] Acknowledge the full author list 2019-11-18 09:29:11 -05:00
Julien Chaumond 0477b307c7 [camembert] tokenizer: use additional_special_tokens 2019-11-16 00:11:07 -05:00
Julien Chaumond f9abf73e31 [camembert] realign w/ recent changes 2019-11-16 00:11:07 -05:00
Julien Chaumond 26858f27cb [camembert] Upload to s3 + rename script 2019-11-16 00:11:07 -05:00
Julien Chaumond 155c782a2c [inputs_embeds] All TF models + tests 2019-11-12 11:29:21 -05:00
Julien Chaumond 2aef2f0bbc [common attributes] Fix previous commit for transfo-xl 2019-11-12 11:29:21 -05:00
Julien Chaumond 2f17464266 [common attributes] Slightly sharper test coverage 2019-11-12 11:29:21 -05:00
Julien Chaumond 9d2398fd99 Ooopsie 2019-11-12 11:29:21 -05:00
Julien Chaumond 70d97ddd60 [TF models] Common attributes as per #1721 2019-11-12 11:29:21 -05:00
Julien Chaumond 872403be1c This is not a @property after all 2019-11-12 11:29:21 -05:00
Julien Chaumond dd6b2e05e1 whitespace 2019-11-12 11:29:21 -05:00
Julien ChaumondLysandre DebutJong Wook KimJeff Wu
 		1c542df7e5 Add RoBERTa-based GPT-2 Output Detector from OpenAI 
converted from https://github.com/openai/gpt-2-output-dataset/tree/master/detector

Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
Co-Authored-By: Jong Wook Kim <jongwook@nyu.edu>
Co-Authored-By: Jeff Wu <wuthefwasthat@gmail.com>
 2019-11-06 16:26:31 -05:00
Julien Chaumond 2f3a421018 Fix other PyTorch models 2019-11-06 14:03:47 -05:00
Julien Chaumond d5319793c4 Fix BERT 2019-11-06 14:03:47 -05:00
Julien Chaumond 27e015bd54 [tests] Flag to test on cuda 2019-11-06 14:03:47 -05:00
Julien Chaumond 13d9135fa5 [tests] get rid of warning 
cf. https://docs.pytest.org/en/latest/example/simple.html
 2019-11-06 14:03:47 -05:00
Julien Chaumond f88c104d8f [run_tf_glue] Add comment for context 2019-11-05 19:56:43 -05:00
Julien Chaumond 30968d70af misc doc 2019-11-05 19:06:12 -05:00
Julien Chaumond 7daacf00df Merge pull request #1695 from huggingface/models_inputs_embeds 
model forwards can take an inputs_embeds param
 2019-11-05 09:55:28 -05:00
Julien Chaumond 00337e9687 [inputs_embeds] All PyTorch models 2019-11-05 00:39:18 +00:00
Julien Chaumond 9eddf44b7a docstring + check 2019-11-04 17:19:15 +00:00
Julien Chaumond 8e11de0e86 model forwards can take an inputs_embeds param 2019-11-04 16:56:26 +00:00
Julien Chaumond 93d2fff071 Close #1654 2019-11-01 09:47:38 -04:00
Julien Chaumond 2a5663c280 Merge branch 'mataney-fix_top_k_top_p_filtering' 2019-10-31 18:28:34 +00:00
Julien Chaumond f96ce1c241 [run_generation] Fix generation with batch_size>1 2019-10-31 18:27:11 +00:00
Julien Chaumond 3c1b6f594e Merge branch 'master' into fix_top_k_top_p_filtering 2019-10-31 13:53:51 -04:00
Julien Chaumond ef1b8b2ae5 [CTRL] warn if generation prompt does not start with a control code 
see also https://github.com/salesforce/ctrl/pull/50
 2019-10-22 21:30:32 +00:00
Julien Chaumond 788e632622 [ner] Honor args.overwrite_cache 2019-10-15 09:17:31 +02:00
Julien Chaumond d688af19e5 Update link to swift-coreml-transformers 
cc @lysandrejik
 2019-10-08 16:37:52 -04:00
Julien Chaumond 9e136ff57c Honor args.overwrite_cache (h/t @erenup) 2019-10-04 15:00:56 -04:00
Julien ChaumondandLysandre Debut b350662955 overflowing_tokens do not really make sense here, let's just return a number 
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2019-09-30 16:37:09 -04:00
Julien Chaumond f5bcde0b2f [multiple-choice] Simplify and use tokenizer.encode_plus 2019-09-30 16:04:55 -04:00