thomwolf
33e72b08d5
fix inner dimensions for 3B/11B models
2019-12-13 11:33:05 +01:00
thomwolf
f19dad61c7
fixing XLM conversion tests with dummy input
2019-12-12 14:46:30 +01:00
thomwolf
fafd4c86ec
fix TF 2.0 version of T5 - update conversion script
2019-12-11 13:47:27 +01:00
thomwolf
67a8be8e90
fix backward in tests
2019-12-10 17:50:32 +01:00
thomwolf
f2538c1274
all tests in torch no grad
2019-12-10 16:33:11 +01:00
thomwolf
a5df980c5b
updating distilbert test
2019-12-10 16:01:15 +01:00
thomwolf
7c3a15ace9
Merge branch 'master' into t5
2019-12-10 15:36:54 +01:00
thomwolf
981a5c8c17
updating models urls
2019-12-10 15:36:19 +01:00
Thomas Wolf
e6cff60b4c
Merge pull request #2069 from huggingface/cleaner-pt-tf-conversion
...
clean up PT <=> TF conversion
2019-12-10 15:34:08 +01:00
Rémi Louf
4b82c485de
remove misplaced summarization documentation
2019-12-10 09:13:33 -05:00
thomwolf
8ae1044f80
updating tests and TF 2.0 model
2019-12-10 15:11:07 +01:00
thomwolf
0558c9cb9b
Merge branch 'master' into t5
2019-12-10 12:58:48 +01:00
Thomas Wolf
e57d00ee10
Merge pull request #1984 from huggingface/squad-refactor
...
[WIP] Squad refactor
2019-12-10 11:07:26 +01:00
Thomas Wolf
ecabbf6d28
Merge pull request #2107 from huggingface/encoder-mask-shape
...
create encoder attention mask from shape of hidden states
2019-12-10 10:07:56 +01:00
thomwolf
608a8f5b56
updating tf 2.0 layer_norm to T5 layer norm
2019-12-10 10:01:01 +01:00
Julien Chaumond
1d18930462
Harmonize no_cuda flag with other scripts
2019-12-09 20:37:55 -05:00
Rémi Louf
f7eba09007
clean for release
2019-12-09 20:37:55 -05:00
Rémi Louf
2a64107e44
improve device usage
2019-12-09 20:37:55 -05:00
Rémi Louf
c0707a85d2
add README
2019-12-09 20:37:55 -05:00
Rémi Louf
ade3cdf5ad
integrate ROUGE
2019-12-09 20:37:55 -05:00
Rémi Louf
076602bdc4
prevent BERT weights from being downloaded twice
2019-12-09 20:37:55 -05:00
Rémi Louf
5909f71028
add py-rouge dependency
2019-12-09 20:37:55 -05:00
Rémi Louf
a1994a71ee
simplified model and configuration
2019-12-09 20:37:55 -05:00
Rémi Louf
3a9a9f7861
default output dir to documents dir
2019-12-09 20:37:55 -05:00
Rémi Louf
693606a75c
update the docs
2019-12-09 20:37:55 -05:00
Rémi Louf
c0443df593
remove beam search
2019-12-09 20:37:55 -05:00
Rémi Louf
2403a66598
give transformers API to BertAbs
2019-12-09 20:37:55 -05:00
Rémi Louf
4d18199902
cast bool tensor to long for pytorch < 1.3
2019-12-09 20:37:55 -05:00
Rémi Louf
9f75565ea8
setup training
2019-12-09 20:37:55 -05:00
Rémi Louf
4735c2af07
tweaks to the BeamSearch API
2019-12-09 20:37:55 -05:00
Rémi Louf
ba089c780b
share pretrained embeddings
2019-12-09 20:37:55 -05:00
Rémi Louf
9660ba1cbd
Add beam search
2019-12-09 20:37:55 -05:00
Rémi Louf
1c71ecc880
load the pretrained weights for encoder-decoder
...
We currently save the pretrained_weights of the encoder and decoder in
two separate directories `encoder` and `decoder`. However, for the
`from_pretrained` function to operate with automodels we need to
specify the type of model in the path to the weights.
The path to the encoder/decoder weights is handled by the
`PreTrainedEncoderDecoder` class in the `save_pretrained` function. Sice
there is no easy way to infer the type of model that was initialized for
the encoder and decoder we add a parameter `model_type` to the function.
This is not an ideal solution as it is error prone, and the model type
should be carried by the Model classes somehow.
This is a temporary fix that should be changed before merging.
2019-12-09 20:37:55 -05:00
Rémi Louf
07f4cd73f6
update function to add special tokens
...
Since I started my PR the `add_special_token_single_sequence` function
has been deprecated for another; I replaced it with the new function.
2019-12-09 20:37:55 -05:00
Pierric Cistac
5c877fe94a
fix albert links
2019-12-09 18:53:00 -05:00
Bilal Khan
79526f82f5
Remove unnecessary epoch variable
2019-12-09 16:24:35 -05:00
Bilal Khan
9626e0458c
Add functionality to continue training from last saved global_step
2019-12-09 16:24:35 -05:00
Bilal Khan
2d73591a18
Stop saving current epoch
2019-12-09 16:24:35 -05:00
Bilal Khan
0eb973b0d9
Use saved optimizer and scheduler states if available
2019-12-09 16:24:35 -05:00
Bilal Khan
a03fcf570d
Save tokenizer after each epoch to be able to resume training from a checkpoint
2019-12-09 16:24:35 -05:00
Bilal Khan
f71b1bb05a
Save optimizer state, scheduler state and current epoch
2019-12-09 16:24:35 -05:00
thomwolf
8e651f56b7
fix tf tests
2019-12-09 22:13:57 +01:00
thomwolf
808bb8da7e
fix transfo xl tests
2019-12-09 21:48:34 +01:00
thomwolf
b016dd16c9
fix tests on python 3.5
2019-12-09 21:38:07 +01:00
LysandreJik
2a4ef098d6
Add ALBERT and XLM to SQuAD script
2019-12-09 10:46:47 -05:00
Lysandre Debut
00c4e39581
Merge branch 'master' into squad-refactor
2019-12-09 10:41:15 -05:00
thomwolf
169fea6855
updating T5
2019-12-09 16:25:33 +01:00
Rémi Louf
3520be7824
create encoder attention mask from shape of hidden states
...
We currently create encoder attention masks (when they're not provided)
based on the shape of the inputs to the encoder. This is obviously
wrong; sequences can be of different lengths. We now create the encoder
attention mask based on the batch_size and sequence_length of the
encoder hidden states.
2019-12-09 11:19:45 +01:00
Aymeric Augustin
0cb163865a
Remove pytest dependency. ( #2093 )
2019-12-07 07:46:14 -05:00
Michael Watkins
2670b0d682
Fix bug which lowercases special tokens
2019-12-06 16:15:53 -05:00
thomwolf