c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,063 Commits 171 Branches 49 Tags
56ee2560bedfc471befe438ec253a1804b6eeb48
Commit Graph
4063 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Patrick von Platen 56ee2560be [Longformer] Better handling of global attention mask vs local attention mask (#4672) 
* better api

* improve automatic setting of global attention mask

* fix longformer bug

* fix global attention mask in test

* fix global attn mask flatten

* fix slow tests

* update docstring

* update docs and make more robust

* improve attention mask
 2020-05-29 17:58:42 +02:00
Simon Böhm e2230ba77b Fix BERT example code for NSP and Multiple Choice (#3953) 
Change the example code to use encode_plus since the token_type_id
wasn't being correctly set.
 2020-05-29 11:55:55 -04:00
Zhangyx 3a5d1ea2a5 Fix two bugs: 1. Index of test data of SST-2. 2. Label index of MNLI data. (#4546) 2020-05-29 11:12:24 -04:00
Patrick von Platen 9c17256447 [Longformer] Multiple choice for longformer (#4645) 
* add multiple choice for longformer

* add models to docs

* adapt docstring

* add test to longformer

* add longformer for mc in init and modeling auto

* fix tests
 2020-05-29 13:46:08 +02:00
Iz Beltagy 91487cbb8e [Longformer] fix model name in examples (#4653) 
* fix longformer model names in examples

* a better name for the notebook
 2020-05-29 13:12:35 +02:00
flozi00 b5015a2a0f gpt2 typo (#4629) 
* gpt2 typo

* Add files via upload
 2020-05-28 16:44:43 -04:00
Iz BeltagyandPatrick von Platen fe5cb1a1c8 Adding community notebook (#4642) 
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-05-28 22:35:15 +02:00
Suraj Patil aecaaf73a4 [Community notebooks] add longformer-for-qa notebook (#4652) 2020-05-28 22:27:22 +02:00
Anthony MOI 5e737018e1 Fix add_special_tokens on fast tokenizers (#4531) 2020-05-28 10:54:45 -04:00
Suraj Patil e444648a30 LongformerForTokenClassification (#4638) 2020-05-28 12:48:18 +02:00
Lavanya ShuklaandPatrick von Platen 3cc2c2a150 add 2 colab notebooks (#4505) 
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-05-28 11:18:16 +02:00
Iz Beltagy ef03ae874f [Longformer] more models + model cards (#4628) 
* adding freeze roberta models

* model cards

* lint
 2020-05-28 11:11:05 +02:00
Patrick von PlatenandLysandre Debut 96f57c9ccb [Benchmark] Memory benchmark utils (#4198) 
* improve memory benchmarking

* correct typo

* fix current memory

* check torch memory allocated

* better pytorch function

* add total cached gpu memory

* add total gpu required

* improve torch gpu usage

* update memory usage

* finalize memory tracing

* save intermediate benchmark class

* fix conflict

* improve benchmark

* improve benchmark

* finalize

* make style

* improve benchmarking

* correct typo

* make train function more flexible

* fix csv save

* better repr of bytes

* better print

* fix __repr__ bug

* finish plot script

* rename plot file

* delete csv and small improvements

* fix in plot

* fix in plot

* correct usage of timeit

* remove redundant line

* remove redundant line

* fix bug

* add hf parser tests

* add versioning and platform info

* make style

* add gpu information

* ensure backward compatibility

* finish adding all tests

* Update src/transformers/benchmark/benchmark_args.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Update src/transformers/benchmark/benchmark_args_utils.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* delete csv files

* fix isort ordering

* add out of memory handling

* add better train memory handling

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-05-27 23:22:16 +02:00
Suraj PatilandPatrick von Platen ec4cdfdd05 LongformerForSequenceClassification (#4580) 
* LongformerForSequenceClassification

* better naming x=>hidden_states, fix typo in doc

* Update src/transformers/modeling_longformer.py

* Update src/transformers/modeling_longformer.py

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-05-27 22:30:00 +02:00
Suraj Patil 4402879ee4 [Model Card] model card for longformer-base-4096-finetuned-squadv1 (#4625) 2020-05-27 18:48:03 +02:00
Lysandre DebutandJulien Chaumond 6a17688021 per_device instead of per_gpu/error thrown when argument unknown (#4618) 
* per_device instead of per_gpu/error thrown when argument unknown

* [docs] Restore examples.md symlink

* Correct absolute links so that symlink to the doc works correctly

* Update src/transformers/hf_argparser.py

Co-authored-by: Julien Chaumond <chaumond@gmail.com>

* Warning + reorder

* Docs

* Style

* not for squad

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-27 11:36:55 -04:00
Mehrdad Farahani 1381b6d01d README for HooshvareLab (#4610) 
HooshvareLab/bert-base-parsbert-uncased
 2020-05-27 11:25:36 -04:00
Patrick von Platen 5acb4edf25 Update version command when contributing (#4614) 2020-05-27 17:19:11 +02:00
Darek Kłeczekandkldarek 842588c12f uncased readme (#4608) 
Co-authored-by: kldarek <darekmail>
 2020-05-27 09:50:04 -04:00
Darek Kłeczek ac1a612179 Create README.md (#4607) 
Model card for cased model
 2020-05-27 09:36:20 -04:00
Sam Shleifer 07797c4da4 [testing] LanguageModelGenerationTests require_tf or require_torch (#4616) 2020-05-27 09:10:26 -04:00
Hao Tan a9aa7456ac Add back --do_lower_case to uncased models (#4245) 
The option `--do_lower_case` is currently required by the uncased models (i.e., bert-base-uncased, bert-large-uncased).

Results:
BERT-BASE without --do_lower_case:  'exact': 73.83, 'f1': 82.22
BERT-BASE with --do_lower_case:  'exact': 81.02, 'f1': 88.34
 2020-05-26 21:13:07 -04:00
Bayartsogt Yadamsuren a801c7fd74 Creating a readme for ALBERT in Mongolian (#4603) 
Here I am uploading Mongolian masked language model (ALBERT) on your platform.
https://en.wikipedia.org/wiki/Mongolia
 2020-05-26 16:54:42 -04:00
Wissam Antoun 6458c0e268 updated model cards for both models at aubmindlab (#4604) 
* updated aubmindlab/bert-base-arabert/ Model card

* updated aubmindlab/bert-base-arabertv01 model card
 2020-05-26 16:52:43 -04:00
Oleksandr Bushkovskyi ea4e7a53fa Improve model card for Tereveni-AI/gpt2-124M-uk-fiction (#4582) 
Add language metadata, training and evaluation corpora details.
Add example output. Fix inconsistent use of quotes.
 2020-05-26 16:51:40 -04:00
Manuel Romero 937930dcae Create README.md (#4591) 2020-05-26 16:50:08 -04:00
Manuel Romero bac1cc4dc1 Remove MD emojis (#4602) 2020-05-26 16:38:39 -04:00
Patrick von Platen 003c477129 [GPT2, CTRL] Allow input of input_ids and past of variable length (#4581) 
* revert convenience  method

* clean docs a bit
 2020-05-26 19:43:58 +02:00
ohmeowandPatrick von Platen 5ddd8d6531 Add BART fine-tuning summarization community notebook (#4539) 
* adding BART summarization how-to community notebook

* Update notebooks/README.md

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-05-26 16:43:41 +02:00
Bram Vanroy 8cc6807e89 Make transformers-cli cross-platform (#4131) 
* make transformers-cli cross-platform

Using "scripts" is a useful option in setup.py particularly when you want to get access to non-python scripts. However, in this case we want to have an entry point into some of our own Python scripts. To do this in a concise, cross-platfom way, we can use entry_points.console_scripts. This change is necessary to provide the CLI on different platforms, which "scripts" does not ensure. Usage remains the same, but the "transformers-cli" script has to be moved (be part of the library) and renamed (underscore + extension)

* make style & quality
 2020-05-26 10:00:51 -04:00
Patrick von Platen c589eae2b8 [Longformer For Question Answering] Conversion script, doc, small fixes (#4593) 
* add new longformer for question answering model

* add new config as well

* fix links

* fix links part 2
 2020-05-26 14:58:47 +02:00
ZhuBaohe a163c9ca5b [T5] Fix Cross Attention position bias (#4499) 
* fix

* fix1
 2020-05-26 08:57:24 -04:00
ZhuBaohe 1d69028989 fix (#4410) 2020-05-26 08:51:28 -04:00
Sam Shleifer b86e42e0ac [ci] fix 3 remaining slow GPU failures (#4584) 2020-05-25 19:20:50 -04:00
Julien Chaumond 365d452d4d [ci] Slow GPU tests run daily (#4465) 2020-05-25 17:28:02 -04:00
Patrick von Platen 3e3e552125 [Reformer] fix reformer num buckets (#4564) 
* fix reformer num buckets

* fix

* adapt docs

* set num buckets in config
 2020-05-25 16:04:45 -04:00
Elman Mansimov 3dea40b858 fixing tokenization of extra_id symbols in T5Tokenizer. Related to issue 4021 (#4353) 2020-05-25 16:04:30 -04:00
Suraj Patil 5139733623 LongformerTokenizerFast (#4547) 2020-05-25 16:03:55 -04:00
Oliver Guhr c9c385c522 Updated the link to the paper (#4570) 
I looks like the conference has changed the link to the paper.
 2020-05-25 15:29:50 -04:00
Sho Arora adab7f8332 Add nn.Module as superclass (#4533) 2020-05-25 15:29:33 -04:00
Manuel Romero 8f7c1c7672 Create model card (#4578) 2020-05-25 15:28:30 -04:00
Ali Safaya 4c6b218056 Update README.md (#4556) 2020-05-25 15:12:23 -04:00
Antonis Maronikolakis 50d1ce411f add DistilBERT to supported models (#4558) 2020-05-25 14:50:45 -04:00
Suraj PatilandPatrick von Platen 03d8527de0 Longformer for question answering (#4500) 
* added LongformerForQuestionAnswering

* add LongformerForQuestionAnswering

* fix import for LongformerForMaskedLM

* add LongformerForQuestionAnswering

* hardcoded sep_token_id

* compute attention_mask if not provided

* combine global_attention_mask with attention_mask when provided

* update example in  docstring

* add assert error messages, better attention combine

* add test for longformerForQuestionAnswering

* typo

* cast gloabl_attention_mask to long

* make style

* Update src/transformers/configuration_longformer.py

* Update src/transformers/configuration_longformer.py

* fix the code quality

* Merge branch 'longformer-for-question-answering' of https://github.com/patil-suraj/transformers into longformer-for-question-answering

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-05-25 18:43:36 +02:00
Bharat Raghunathan a34a9896ac DOC: Fix typos in modeling_auto (#4534) 2020-05-23 09:40:59 -04:00
Bijay GurungandJulien Chaumond e19b978151 Add Type Hints to modeling_utils.py Closes #3911 (#3948) 
* Add Type Hints to modeling_utils.py Closes #3911

Add Type Hints to methods in `modeling_utils.py`

Note: The coverage isn't 100%. Mostly skipped internal methods.

* Reformat according to `black` and `isort`

* Use typing.Iterable instead of Sequence

* Parameterize Iterable by its generic type

* Use typing.Optional when None is the default value

* Adhere to style guideline

* Update src/transformers/modeling_utils.py

* Update src/transformers/modeling_utils.py

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-05-22 19:10:22 -04:00
Funtowicz Morgan 996f393a86 Warn the user about max_len being on the path to be deprecated. (#4528) 
* Warn the user about max_len being on the path to be deprecated.

* Ensure better backward compatibility when max_len is provided to a tokenizer.

* Make sure to override the parameter and not the actual instance value.

* Format & quality
 2020-05-22 18:08:30 -04:00
Patrick von Platen 0f6969b7e9 Better github link for Reformer Colab Notebook 2020-05-22 23:51:36 +02:00
Sam Shleifer ab44630db2 [Summarization Pipeline]: Fix default tokenizer (#4506) 
* Fix pipelines defaults bug

* one liner

* style
 2020-05-22 17:49:45 -04:00
Julien Chaumond 2c1ebb8b50 Re-apply #4446 + add packaging dependency 
As discussed w/ @lysandrejik

packaging is maintained by PyPA (the Python Packaging Authority), and should be lightweight and stable
 2020-05-22 17:29:03 -04:00