 Pascal Voitot
|
5b7b78e088
|
🐛 #2096 in tokenizer.decode, adds a space after special tokens to return right formatted string
|
2019-12-13 16:41:44 -05:00 |
|
|
Julien Chaumond
|
866d73ca26
|
[cli] Upload is now compatible with folders
|
2019-12-13 16:39:08 -05:00 |
|
|
Lysandre
|
d461472948
|
return for SQuAD [BLACKED]
|
2019-12-13 15:31:52 -05:00 |
|
|
Lysandre
|
f24a228a93
|
Speed up tokenization process
|
2019-12-13 14:50:35 -05:00 |
|
|
Lysandre
|
c8ed1c82c8
|
[SQUAD] Load checkpoint when evaluating without training
|
2019-12-13 12:13:48 -05:00 |
|
|
thomwolf
|
5c00e344c1
|
update model doc - swith 3B/11B to 3b/11b
|
2019-12-13 16:33:29 +01:00 |
|
|
Morgan Funtowicz
|
0b51532ce9
|
Reintroducing the batch_encode_plus method
|
2019-12-13 16:22:50 +01:00 |
|
|
Thomas Wolf
|
110394b2ba
|
Merge branch 'master' into t5
|
2019-12-13 16:03:32 +01:00 |
|
|
Pierric Cistac
|
5a5c4349e8
|
Fix summarization to_cpu doc
|
2019-12-13 10:02:33 -05:00 |
|
|
thomwolf
|
8ade204098
|
fix tf
|
2019-12-13 14:48:47 +01:00 |
|
|
thomwolf
|
47f0e3cfb7
|
cleaning up configuration classes
|
2019-12-13 14:33:24 +01:00 |
|
|
Morgan Funtowicz
|
8938b546bf
|
Removed from_config
|
2019-12-13 14:27:04 +01:00 |
|
|
Morgan Funtowicz
|
1ca52567a4
|
Allow model conversion in the pipeline allocator.
|
2019-12-13 14:13:14 +01:00 |
|
|
Morgan Funtowicz
|
28e64ad5a4
|
Raise an exception if the pipeline allocator can't determine the tokenizer from the model.
|
2019-12-13 14:12:54 +01:00 |
|
|
Morgan Funtowicz
|
be5bf7b81b
|
Added NER pipeline.
|
2019-12-13 14:12:17 +01:00 |
|
|
Morgan Funtowicz
|
80eacb8f16
|
Adding labels mapping for classification models in their respective config.
|
2019-12-13 14:10:22 +01:00 |
|
|
thomwolf
|
33e72b08d5
|
fix inner dimensions for 3B/11B models
|
2019-12-13 11:33:05 +01:00 |
|
|
erenup
|
9b312f9d41
|
initial version for roberta squad
|
2019-12-13 14:51:40 +08:00 |
|
|
erenup
|
40ed717232
|
Merge remote-tracking branch 'refs/remotes/huggingface/master'
|
2019-12-13 09:10:17 +08:00 |
|
|
LysandreJik
|
7296f1010b
|
Cleanup squad and add allow train_file and predict_file usage
|
2019-12-12 13:01:04 -05:00 |
|
|
Julien Chaumond
|
5d67aa21ae
|
[doc] Replicate doc from #2144
|
2019-12-12 12:39:41 -05:00 |
|
|
LysandreJik
|
3fd71c4431
|
Update example scripts
|
2019-12-12 12:08:54 -05:00 |
|
|
LysandreJik
|
fe92755b99
|
Fix special tokens mask in encode
|
2019-12-12 11:37:19 -05:00 |
|
|
Alan deLevie
|
fbf5455a86
|
Fix typo in examples/run_glue.py args declaration.
deay -> decay
|
2019-12-12 11:16:19 -05:00 |
|
|
thomwolf
|
f19dad61c7
|
fixing XLM conversion tests with dummy input
|
2019-12-12 14:46:30 +01:00 |
|
|
Morgan Funtowicz
|
f69dbecc38
|
Expose classification labels mapping (and reverse) in model config.
|
2019-12-12 10:25:36 +01:00 |
|
|
Thomas Wolf
|
90df44f0aa
|
Merge pull request #2063 from guillaume-be/special_tokens_mask_value_not_used
special_tokens_mask value was unused and calculated twice
|
2019-12-12 08:21:46 +01:00 |
|
|
Thomas Wolf
|
707f9e9241
|
Merge pull request #2081 from pglock/patch-1
handle string with only whitespaces as empty
|
2019-12-12 08:20:43 +01:00 |
|
|
Thomas Wolf
|
137e20a846
|
Merge pull request #2075 from huggingface/check-link-validity
Check link validity
|
2019-12-12 08:09:12 +01:00 |
|
|
Thomas Wolf
|
d5712f7cac
|
Merge branch 'master' into check-link-validity
|
2019-12-12 08:00:51 +01:00 |
|
|
Thomas Wolf
|
9c58b236ef
|
Merge pull request #2144 from huggingface/from-pretrained-from-url
Allowing from_pretrained to load from url directly
|
2019-12-12 07:43:40 +01:00 |
|
|
thomwolf
|
413f41921b
|
fix merge
|
2019-12-12 07:34:42 +01:00 |
|
|
Thomas Wolf
|
386a93f0f8
|
Merge branch 'master' into from-pretrained-from-url
|
2019-12-12 07:31:05 +01:00 |
|
|
Thomas Wolf
|
2d103546ef
|
Merge pull request #2148 from huggingface/fix_encode_plus
Fix encode plus
|
2019-12-12 07:24:47 +01:00 |
|
|
Julien Chaumond
|
1748fdf657
|
[doc] Fix rst table
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
36fc52a3b4
|
Update links to weights
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
371c5ddfad
|
Py2 tests for Lysandre
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
5505cf7014
|
Run tests on Py2 too, for Lysandre
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
9cb97c0c0f
|
Actually run the tests
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
95854c4a2f
|
Actually run the tests
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
d2100428d3
|
Update to new test infra and only run conditionally
|
2019-12-11 18:32:27 -05:00 |
|
|
Masatoshi Suzuki
|
597ba7feb3
|
Support testing Japanese BERT tokenizers
|
2019-12-11 18:32:27 -05:00 |
|
|
Masatoshi Suzuki
|
6a43dc9d7d
|
Support Python 2
|
2019-12-11 18:32:27 -05:00 |
|
|
Masatoshi Suzuki
|
a09da4eeb0
|
Add a test for Japanese BERT tokenizers
|
2019-12-11 18:32:27 -05:00 |
|
|
Masatoshi Suzuki
|
57b5cb3eaa
|
Fix loading BertJapaneseTokenizer
|
2019-12-11 18:32:27 -05:00 |
|
|
Masatoshi Suzuki
|
c03c0dfd23
|
Add support for Japanese BERT models by cl-tohoku
|
2019-12-11 18:32:27 -05:00 |
|
|
Julien Chaumond
|
4f15e5a267
|
Add tests.
Maybe not the best possible place for the tests, lmk.
|
2019-12-11 17:41:51 -05:00 |
|
|
Julien Chaumond
|
18e1f751f1
|
TF support
|
2019-12-11 17:07:46 -05:00 |
|
|
Julien Chaumond
|
31e5b5ff22
|
Fix tests + first example of doc
|
2019-12-11 15:22:02 -05:00 |
|
|
LysandreJik
|
3d57c51111
|
Fix encode plus
|
2019-12-11 15:10:17 -05:00 |
|