Julien Chaumond
2aef2f0bbc
[common attributes] Fix previous commit for transfo-xl
2019-11-12 11:29:21 -05:00
Julien Chaumond
2f17464266
[common attributes] Slightly sharper test coverage
2019-11-12 11:29:21 -05:00
Julien Chaumond
9d2398fd99
Ooopsie
2019-11-12 11:29:21 -05:00
Julien Chaumond
70d97ddd60
[TF models] Common attributes as per #1721
2019-11-12 11:29:21 -05:00
Julien Chaumond
872403be1c
This is not a @property after all
2019-11-12 11:29:21 -05:00
Julien Chaumond
dd6b2e05e1
whitespace
2019-11-12 11:29:21 -05:00
Lysandre
d409aca326
Clarify the use of past in GPT2 and CTRL
2019-11-12 10:59:37 -05:00
Michael Watkins
7246d3c2f9
Consider do_lower_case in PreTrainedTokenizer
...
As pointed out in #1545 , when using an uncased model, and adding
a new uncased token, the tokenizer does not correctly identify this
in the case that the input text contains the token in a cased format.
For instance, if we load bert-base-uncased into BertTokenizer, and
then use .add_tokens() to add "cool-token", we get the expected
result for .tokenize('this is a cool-token'). However, we get a
possibly unexpected result for .tokenize('this is a cOOl-Token'),
which in fact mirrors the result for the former from before the new
token was added.
This commit adds
- functionality to PreTrainedTokenizer to handle this
situation in case a tokenizer (currently Bert, DistilBert,
and XLNet) has the do_lower_case=True kwarg by:
1) lowercasing tokens added with .add_tokens()
2) lowercasing text at the beginning of .tokenize()
- new common test case for tokenizers
https://github.com/huggingface/transformers/issues/1545
2019-11-12 13:08:30 +02:00
ronakice
2e31176557
fix multi-gpu eval
2019-11-12 05:55:11 -05:00
thomwolf
8aba81a0b6
fix #1789
2019-11-12 08:52:43 +01:00
Stefan Schweter
94e55253ae
tests: add test case for DistilBertForTokenClassification implementation
2019-11-11 16:20:15 +01:00
Stefan Schweter
2b07b9e5ee
examples: add DistilBert support for NER fine-tuning
2019-11-11 16:19:34 +01:00
Stefan Schweter
1806eabf59
module: add DistilBertForTokenClassification import
2019-11-11 16:18:48 +01:00
Stefan Schweter
1c7253cc5f
modeling: add DistilBertForTokenClassification implementation
2019-11-11 16:18:16 +01:00
Lysandre
b5d330d118
Fix #1784
2019-11-11 10:15:14 -05:00
eukaryote
90f6e73a35
Add DialoGPT support for Pytorch->TF
2019-11-09 16:46:19 +00:00
eukaryote
ef99852961
from_pretrained: convert DialoGPT format
...
DialoGPT checkpoints have "lm_head.decoder.weight" instead of "lm_head.weight".
(see: https://www.reddit.com/r/MachineLearning/comments/dt5woy/p_dialogpt_state_of_the_art_conversational_model/f6vmwuy?utm_source=share&utm_medium=web2x )
2019-11-09 16:32:40 +00:00
Adrian Bauer
7a9aae1044
Fix run_bertology.py
...
Make imports and args.overwrite_cache match run_glue.py
2019-11-08 16:28:40 -05:00
Rémi Louf
cd286c2145
add condition around mask transformation
2019-11-08 11:31:16 +01:00
Rémi Louf
28d0ba35d7
only init encoder_attention_mask if stack is decoder
...
We currently initialize `encoder_attention_mask` when it is `None`,
whether the stack is that of an encoder or a decoder. Since this
may lead to bugs that are difficult to tracks down, I added a condition
that assesses whether the current stack is a decoder.
2019-11-08 11:22:19 +01:00
Diganta Misra
070dcf1c02
Added Mish Activation Function
...
Mish is a new activation function proposed here - https://arxiv.org/abs/1908.08681
It has seen some recent success and has been adopted in SpaCy, Thic, TensorFlow Addons and FastAI-dev.
All benchmarks recorded till now (including against ReLU, Swish and GELU) is present in the repository - https://github.com/digantamisra98/Mish
Might be a good addition to experiment with especially in the Bert Model.
2019-11-07 03:45:43 +05:30
1c542df7e5
Add RoBERTa-based GPT-2 Output Detector from OpenAI
...
converted from https://github.com/openai/gpt-2-output-dataset/tree/master/detector
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr >
Co-Authored-By: Jong Wook Kim <jongwook@nyu.edu >
Co-Authored-By: Jeff Wu <wuthefwasthat@gmail.com >
2019-11-06 16:26:31 -05:00
Julien Chaumond
2f3a421018
Fix other PyTorch models
2019-11-06 14:03:47 -05:00
Julien Chaumond
d5319793c4
Fix BERT
2019-11-06 14:03:47 -05:00
Julien Chaumond
27e015bd54
[tests] Flag to test on cuda
2019-11-06 14:03:47 -05:00
Julien Chaumond
13d9135fa5
[tests] get rid of warning
...
cf. https://docs.pytest.org/en/latest/example/simple.html
2019-11-06 14:03:47 -05:00
Julien Chaumond
f88c104d8f
[run_tf_glue] Add comment for context
2019-11-05 19:56:43 -05:00
Julien Chaumond
30968d70af
misc doc
2019-11-05 19:06:12 -05:00
Dom Hudson
de890ae67d
Updating docblocks in optimizers.py
2019-11-05 17:31:29 -05:00
Lysandre
d7d36181fd
GPT-2 XL
2019-11-05 13:31:58 -05:00
LysandreJik
151e4ab4e7
Fix CTRL past
2019-11-05 16:26:51 +00:00
Julien Chaumond
7daacf00df
Merge pull request #1695 from huggingface/models_inputs_embeds
...
model forwards can take an inputs_embeds param
2019-11-05 09:55:28 -05:00
Clement
a44f112fb9
add authors for models
2019-11-05 08:48:26 -05:00
Filip Povolny
124409d075
Make dummy inputs a property of TFPreTrainedModel.
2019-11-05 11:48:45 +01:00
Thomas Wolf
e99071f105
Merge pull request #1734 from orena1/patch-1
...
add progress bar to convert_examples_to_features
2019-11-05 11:34:20 +01:00
Thomas Wolf
ba973342e3
Merge pull request #1553 from WilliamTambellini/timeSquadInference
...
Add speed log to examples/run_squad.py
2019-11-05 11:13:12 +01:00
Filip Povolny
8df7dfd2a7
Make dummy inputs a local variable in TFPreTrainedModel.
2019-11-05 11:09:16 +01:00
Thomas Wolf
237fad339c
Merge pull request #1709 from oneraghavan/master
...
Fixing mode in evaluate during training
2019-11-05 10:55:33 +01:00
thomwolf
f1e4db2aa8
Fix #1686
2019-11-05 09:38:00 +01:00
Oren Amsalem
d7906165a3
add progress bar for convert_examples_to_features
...
It takes considerate amount of time (~10 min) to parse the examples to features, it is good to have a progress-bar to track this
2019-11-05 10:34:27 +02:00
Thomas Wolf
d2e2577dd3
Merge pull request #1723 from huggingface/fix-1623
...
Fix #1623
2019-11-05 08:36:30 +01:00
Julien Chaumond
00337e9687
[inputs_embeds] All PyTorch models
2019-11-05 00:39:18 +00:00
Julien Chaumond
9eddf44b7a
docstring + check
2019-11-04 17:19:15 +00:00
Julien Chaumond
8e11de0e86
model forwards can take an inputs_embeds param
2019-11-04 16:56:26 +00:00
Lysandre and Santosh-Gupta
68f7064a3e
Add model.train() line to ReadMe training example
...
Co-Authored-By: Santosh-Gupta <San.Gupta.ML@gmail.com >
2019-11-04 11:52:35 -05:00
thomwolf
8d6b9d717c
fix #1532 and encode_plus
2019-11-04 17:07:51 +01:00
Thomas Wolf
c8f2712199
Merge pull request #1721 from huggingface/common_attributes
...
Add common getter and setter for input_embeddings & output_embeddings
2019-11-04 16:21:52 +01:00
thomwolf
89d6272898
Fix #1623
2019-11-04 16:21:12 +01:00
thomwolf
b340a910ed
fix tests - flagged as slow all the tests downloading from AWS
2019-11-04 16:03:36 +01:00
thomwolf
f02805da6f
fix tests
2019-11-04 15:42:23 +01:00
Julien Chaumond