3 people 8ff88d25e9 [fsmt] rewrite SinusoidalPositionalEmbedding + USE_CUDA test fixes + new TranslationPipeline test (#7224) ...

* fix USE_CUDA, add pipeline

* USE_CUDA fix

* recode SinusoidalPositionalEmbedding into nn.Embedding subclass

was needed for torchscript to work - this is now part of the state_dict, so will have to remove these keys during save_pretrained

* back out (ci debug)

* restore

* slow last?

* facilitate not saving certain keys and test

* remove no longer used keys

* style

* fix logging import

* cleanup

* Update src/transformers/modeling_utils.py

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

* fix bug in max_positional_embeddings

* rename keys to keys_to_never_save per suggestion, improve the setup

* Update src/transformers/modeling_utils.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

2020-09-21 09:13:35 -04:00

..

fixtures

Albert pretrain datasets/ datacollator (#6168)

2020-09-10 07:56:29 -04:00

__init__.py

GPU text generation: mMoved the encoded_prompt to correct device

2020-01-06 15:11:12 +01:00

conftest.py

ignore FutureWarning in tests (#7079)

2020-09-14 07:50:51 -04:00

test_activations_tf.py

Refactoring the TF activations functions (#7150)

2020-09-16 07:03:47 -04:00

test_activations.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_benchmark_tf.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_benchmark.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_cli.py

[transformers-cli] fix logger getter (#6777)

2020-08-27 20:01:17 -04:00

test_configuration_auto.py

Move tests/utils.py -> transformers/testing_utils.py (#5350)

2020-07-01 10:31:17 -04:00

test_configuration_common.py

Pass kwargs to configuration (#3147)

2020-03-05 17:16:57 -05:00

test_data_collator.py

Albert pretrain datasets/ datacollator (#6168)

2020-09-10 07:56:29 -04:00

test_doc_samples.py

Move tests/utils.py -> transformers/testing_utils.py (#5350)

2020-07-01 10:31:17 -04:00

test_hf_api.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_hf_argparser.py

parse arguments from dict (#4869)

2020-07-31 04:44:23 -04:00

test_logging.py

adding TRANSFORMERS_VERBOSITY env var (#6961)

2020-09-09 04:08:01 -04:00

test_model_card.py

GPU text generation: mMoved the encoded_prompt to correct device

2020-01-06 15:11:12 +01:00

test_model_output.py

Add tests and fix various bugs in ModelOutput (#7073)

2020-09-11 12:01:33 -04:00

test_modeling_albert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_auto.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_bart.py

prepare_seq2seq_batch makes labels/ decoder_input_ids made later. (#6654)

2020-08-28 11:15:17 -04:00

test_modeling_bert_generation.py

Add "Leveraging Pretrained Checkpoints for Generation" Seq2Seq models. (#6594)

2020-09-10 16:40:51 +02:00

test_modeling_bert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_camembert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_common.py

Funnel transformer (#6908)

2020-09-08 08:08:08 -04:00

test_modeling_ctrl.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_distilbert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_dpr.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_electra.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_encoder_decoder.py

clean naming (#7068)

2020-09-11 09:57:53 +02:00

test_modeling_flaubert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_fsmt.py

[fsmt] rewrite SinusoidalPositionalEmbedding + USE_CUDA test fixes + new TranslationPipeline test (#7224)

2020-09-21 09:13:35 -04:00

test_modeling_funnel.py

Funnel transformer (#6908)

2020-09-08 08:08:08 -04:00

test_modeling_gpt2.py

[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)

2020-09-01 12:38:25 +02:00

test_modeling_longformer.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_lxmert.py

fix (#6946)

2020-09-04 16:08:54 +02:00

test_modeling_marian.py

Fix marian slow test (#6854)

2020-08-31 16:10:43 -04:00

test_modeling_mbart.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_mobilebert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_openai.py

[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)

2020-09-01 12:38:25 +02:00

test_modeling_pegasus.py

Pegasus finetune script: add --adafactor (#6811)

2020-08-29 17:43:32 -04:00

test_modeling_reformer.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_roberta.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_t5.py

[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)

2020-09-01 12:38:25 +02:00

test_modeling_tf_albert.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_tf_auto.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_tf_bert.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_tf_camembert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_tf_common.py

Add TF Funnel Transformer (#7029)

2020-09-10 10:41:56 -04:00

test_modeling_tf_ctrl.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_tf_distilbert.py

test_tf_common: remove un_used mixin class parameters (#6866)

2020-09-02 10:54:40 -04:00

test_modeling_tf_electra.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_tf_flaubert.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_tf_funnel.py

Add TF Funnel Transformer (#7029)

2020-09-10 10:41:56 -04:00

test_modeling_tf_gpt2.py

[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)

2020-09-01 12:38:25 +02:00

test_modeling_tf_longformer.py

test_tf_common: remove un_used mixin class parameters (#6866)

2020-09-02 10:54:40 -04:00

test_modeling_tf_lxmert.py

Adding the LXMERT pretraining model (MultiModal languageXvision) to HuggingFace's suite of models (#5793)

2020-09-03 04:02:25 -04:00

test_modeling_tf_mobilebert.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_tf_openai.py

[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)

2020-09-01 12:38:25 +02:00

test_modeling_tf_roberta.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_tf_t5.py

[Generate] Facilitate PyTorch generate using ModelOutputs (#6735)

2020-09-01 12:38:25 +02:00

test_modeling_tf_transfo_xl.py

test_tf_common: remove un_used mixin class parameters (#6866)

2020-09-02 10:54:40 -04:00

test_modeling_tf_xlm_roberta.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_tf_xlm.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_tf_xlnet.py

test_tf_common: remove un_used mixin class parameters (#6866)

2020-09-02 10:54:40 -04:00

test_modeling_transfo_xl.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_xlm_roberta.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_modeling_xlm.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_modeling_xlnet.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_onnx.py

Fix flaky ONNX tests (#6531)

2020-08-17 09:04:35 -04:00

test_optimization_tf.py

Update repo to isort v5 (#6686)

2020-08-24 11:03:01 -04:00

test_optimization.py

Format

2020-08-27 18:31:51 +02:00

test_pipelines.py

feat: allow prefix for any generative model (#5885)

2020-09-07 03:03:45 -04:00

test_tokenization_albert.py

[HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510)

2020-06-15 17:12:51 -04:00

test_tokenization_auto.py

[from_pretrained] Allow tokenizer_type ≠ model_type (#6995)

2020-09-09 04:22:59 -04:00

test_tokenization_bart.py

[tests] fix typos in inputs (#6818)

2020-08-30 18:19:57 +08:00

test_tokenization_bert_generation.py

clean naming (#7068)

2020-09-11 09:57:53 +02:00

test_tokenization_bert_japanese.py

Support additional dictionaries for BERT Japanese tokenizers (#6515)

2020-08-17 12:00:23 +08:00

test_tokenization_bert.py

Add strip_accents to basic BertTokenizer. (#6280)

2020-08-06 18:52:28 +08:00

test_tokenization_bertweet.py

Add new pre-trained models BERTweet and PhoBERT (#6129)

2020-09-18 13:16:43 -04:00

test_tokenization_common.py

[QOL] add signature for prepare_seq2seq_batch (#7108)

2020-09-14 20:33:08 -04:00

test_tokenization_ctrl.py

[HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510)

2020-06-15 17:12:51 -04:00

test_tokenization_distilbert.py

Move tests/utils.py -> transformers/testing_utils.py (#5350)

2020-07-01 10:31:17 -04:00

test_tokenization_dpr.py

Fix tests imports dpr (#5576)

2020-07-07 16:35:12 +02:00

test_tokenization_fast.py

Fix style (#6803)

2020-08-28 15:02:25 -04:00

test_tokenization_fsmt.py

[ported model] FSMT (FairSeq MachineTranslation) (#6940)

2020-09-17 11:31:29 -04:00

test_tokenization_funnel.py

Funnel transformer (#6908)

2020-09-08 08:08:08 -04:00

test_tokenization_gpt2.py

[HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510)

2020-06-15 17:12:51 -04:00

test_tokenization_lxmert.py

Adding the LXMERT pretraining model (MultiModal languageXvision) to HuggingFace's suite of models (#5793)

2020-09-03 04:02:25 -04:00

test_tokenization_marian.py

rename prepare_translation_batch -> prepare_seq2seq_batch (#6103)

2020-08-11 15:57:07 -04:00

test_tokenization_mbart.py

prepare_seq2seq_batch makes labels/ decoder_input_ids made later. (#6654)

2020-08-28 11:15:17 -04:00

test_tokenization_openai.py

[HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510)

2020-06-15 17:12:51 -04:00

test_tokenization_pegasus.py

prepare_seq2seq_batch makes labels/ decoder_input_ids made later. (#6654)

2020-08-28 11:15:17 -04:00

test_tokenization_phobert.py

Add new pre-trained models BERTweet and PhoBERT (#6129)

2020-09-18 13:16:43 -04:00

test_tokenization_reformer.py

Black 20 release

2020-08-26 17:20:22 +02:00

test_tokenization_roberta.py

prepare_seq2seq_batch makes labels/ decoder_input_ids made later. (#6654)

2020-08-28 11:15:17 -04:00

test_tokenization_t5.py

[T5Tokenizer] remove prefix_tokens (#7078)

2020-09-11 14:18:45 -04:00

test_tokenization_transfo_xl.py

Transformer-XL: Improved tokenization with sacremoses (#6322)

2020-08-28 09:56:17 -04:00

test_tokenization_utils.py

Fixes to make life easier with the nlp library (#6423)

2020-08-12 08:00:56 -04:00

test_tokenization_xlm_roberta.py

Move tests/utils.py -> transformers/testing_utils.py (#5350)

2020-07-01 10:31:17 -04:00

test_tokenization_xlm.py

Move tests/utils.py -> transformers/testing_utils.py (#5350)

2020-07-01 10:31:17 -04:00

test_tokenization_xlnet.py

Move tests/utils.py -> transformers/testing_utils.py (#5350)

2020-07-01 10:31:17 -04:00

test_trainer_distributed.py

Add tests to Trainer (#6605)

2020-08-20 11:13:50 -04:00

test_trainer.py

Trainer multi label (#7191)

2020-09-17 08:15:37 -04:00