c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,145 Commits 171 Branches 49 Tags
9629e2c676efab31d1f27a3f27d811e91fd05a92
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Thomas Wolf 9629e2c676 Merge pull request #1804 from ronakice/master 
fix multi-gpu eval in torch examples
 2019-11-14 22:24:05 +01:00
Thomas Wolf 5b322a36db Merge pull request #1811 from huggingface/special-tokens 
Fix special tokens addition in decoder #1807
 2019-11-14 22:17:24 +01:00
Thomas Wolf 1a237d7f42 Merge pull request #1831 from iedmrc/gpt2-tokenization-sum-func-replacement 
sum() is replaced by itertools.chain.from_iterable()
 2019-11-14 22:11:54 +01:00
Thomas Wolf df99f8c5a1 Merge pull request #1832 from huggingface/memory-leak-schedulers 
replace LambdaLR scheduler wrappers by function
 2019-11-14 22:10:31 +01:00
Thomas Wolf 0be9ae7b3e Merge pull request #1833 from huggingface/max-length-warning 
Token indices sequence length is longer than the specified maximum sequence length for this model
 2019-11-14 22:04:49 +01:00
Thomas Wolf e99071f105 Merge pull request #1734 from orena1/patch-1 
add progress bar to convert_examples_to_features
 2019-11-05 11:34:20 +01:00
Thomas Wolf ba973342e3 Merge pull request #1553 from WilliamTambellini/timeSquadInference 
Add speed log to examples/run_squad.py
 2019-11-05 11:13:12 +01:00
Thomas Wolf 237fad339c Merge pull request #1709 from oneraghavan/master 
Fixing mode in evaluate during training
 2019-11-05 10:55:33 +01:00
Thomas Wolf d2e2577dd3 Merge pull request #1723 from huggingface/fix-1623 
Fix #1623
 2019-11-05 08:36:30 +01:00
Thomas Wolf c8f2712199 Merge pull request #1721 from huggingface/common_attributes 
Add common getter and setter for input_embeddings & output_embeddings
 2019-11-04 16:21:52 +01:00
Thomas Wolf 1d4d070256 Merge pull request #1549 from hlums/master 
Fix token order in xlnet preprocessing for SQuAD
 2019-11-04 15:37:15 +01:00
Thomas Wolf 9a3b173cd3 Merge branch 'master' into master 2019-11-04 11:41:26 +01:00
Thomas Wolf 8a62835577 Merge pull request #1679 from cregouby/master 
Fix https://github.com/huggingface/transformers/issues/1673
 2019-11-01 22:02:24 +01:00
Thomas Wolf c7058d8224 Merge pull request #1608 from focox/master 
Error raised by "tmp_eval_loss += tmp_eval_loss.item()" when using multi-gpu
 2019-10-30 17:14:07 +01:00
Thomas Wolf 22838f19fd Merge pull request #1668 from tlkh/fix-tf-xlm 
Fixed training for TF XLM
 2019-10-30 17:08:00 +01:00
Thomas Wolf 7f84fc571a Merge pull request #1670 from huggingface/templates 
Templates and explanation for adding a new model and example script
 2019-10-30 17:05:58 +01:00
Thomas Wolf 04c69db399 Merge pull request #1628 from huggingface/tfglue 
run_tf_glue works with all tasks
 2019-10-30 17:04:03 +01:00
Thomas Wolf 5c6a19a94a Merge pull request #1604 from huggingface/deploy_doc 
Versioning in documentation
 2019-10-30 17:03:14 +01:00
Thomas Wolf 3df4367244 Merge pull request #1601 from huggingface/clean-roberta 
Clean roberta model & all tokenizers now add special tokens by default (breaking change)
 2019-10-30 17:00:40 +01:00
Thomas Wolf 6d73c92cae Merge pull request #1455 from huggingface/conditional-generation 
[WIP] Sequence generation using pretrained BERT
 2019-10-30 16:54:18 +01:00
Thomas Wolf 36174696cc Merge branch 'master' into clean-roberta 2019-10-30 16:51:06 +01:00
Thomas Wolf 228cdd6a6e Merge branch 'master' into conditional-generation 2019-10-30 16:40:35 +01:00
Thomas WolfandStefan Schweter 55fbfea369 Update CONTRIBUTING.md 
Co-Authored-By: Stefan Schweter <stefan.schweter@bsb-muenchen.de>
 2019-10-30 12:25:40 +01:00
Thomas WolfandStefan Schweter cef2a8f900 Update CONTRIBUTING.md 
Co-Authored-By: Stefan Schweter <stefan.schweter@bsb-muenchen.de>
 2019-10-30 12:25:31 +01:00
Thomas Wolf b8c9ea0010 Merge pull request #1580 from pminervini/master 
Gradient norm clipping should be done right before calling the optimiser
 2019-10-22 13:59:20 +02:00
Thomas Wolf 0e64fec1ab Merge pull request #1568 from daemon/patch-1 
Fix hanging when loading pretrained models
 2019-10-21 14:31:57 +02:00
Thomas Wolf 40f14ff545 Merge pull request #1513 from slayton58/amp_fp16_einsum 
Force einsum to run in fp16
 2019-10-15 10:25:00 +02:00
Thomas Wolf e703e4dfe1 Merge pull request #1509 from julian-pani/patch-3 
remove leftover usage of DUMMY_INPUTS
 2019-10-15 10:24:13 +02:00
Thomas Wolf d147671c6c Merge pull request #1508 from tlkh/master 
Added performance enhancements (XLA, AMP) to examples
 2019-10-15 09:57:18 +02:00
Thomas Wolf 08bd8f9f39 Merge pull request #1505 from e-budur/master 
Fixed the sample code in the title 'Quick tour'.
 2019-10-15 09:50:36 +02:00
Thomas Wolf 8aa3b753bd Merge pull request #1434 from bryant1410/patch-1 
Remove unnecessary use of FusedLayerNorm in XLNet
 2019-10-15 09:44:19 +02:00
Thomas Wolf 621e7a2529 Merge pull request #1275 from stecklin/ner-fine-tuning 
Implement fine-tuning BERT on CoNLL-2003 named entity recognition task
 2019-10-15 09:35:24 +02:00
Thomas Wolf 80889a0226 Merge pull request #1512 from louismartin/fix-roberta-convert 
Fix import error in script to convert faisreq roberta checkpoints
 2019-10-14 17:40:32 +02:00
Thomas Wolf f62f992cf7 Merge pull request #1502 from jeffxtang/master 
the working example code to use BertForQuestionAnswering
 2019-10-14 16:14:52 +02:00
Thomas Wolf 4428aefc63 Merge pull request #1488 from huggingface/pytorch-tpu 
GLUE on TPU
 2019-10-11 16:33:00 +02:00
Thomas Wolf 3b43b01872 Merge pull request #1482 from huggingface/tf2_integration_tests 
Integration of TF 2.0 models with other Keras modules
 2019-10-11 16:25:43 +02:00
Thomas Wolf 700331b5ec Merge pull request #1492 from stefan-it/bert-german-dbmdz-models 
Add new BERT models for German (cased and uncased)
 2019-10-11 13:01:52 +02:00
Thomas Wolf 573dde9b44 Merge pull request #1405 from slayton58/xlnet_layer_reorder 
Re-order XLNet attention head outputs for better perf
 2019-10-11 12:10:58 +02:00
Thomas Wolf 6596e3d566 Merge pull request #1454 from bkkaggle/pytorch-built-in-tensorboard 
Change tensorboard imports to use built-in tensorboard if available
 2019-10-10 11:56:55 +02:00
Thomas Wolf 4bc4601192 Merge pull request #1480 from huggingface/fix_ctrl_tokenizer 
Fixing CTRL tokenizer - Update error messages - XLM-MLM in run_generation
 2019-10-10 11:56:20 +02:00
Thomas Wolf 58b302caf3 Merge pull request #1398 from dveselov/patch-1 
Fixed typo in docs README
 2019-10-09 03:52:42 +02:00
Thomas Wolf 439fac723a Merge pull request #1409 from brian41005/master 
Evaluation result.txt path changing #1286
 2019-10-09 03:14:34 +02:00
Thomas Wolf 1615360c71 Merge pull request #1438 from SeanBE/master 
fix pytorch-transformers migration description in README
 2019-10-07 05:02:23 -04:00
Thomas Wolf b3cfd97946 Merge pull request #1373 from TimYagan/fix-css 
Fixed critical css font-family issues
 2019-10-03 19:04:02 -04:00
Thomas Wolf 1569610f2d Merge pull request #1296 from danai-antoniou/add-duplicate-tokens-error 
Added ValueError for duplicates in list of added tokens
 2019-10-03 17:06:17 -04:00
Thomas Wolf 963529e29b Merge pull request #1288 from echan00/master 
Typo with LM Fine tuning script
 2019-10-01 18:46:07 -04:00
Thomas Wolf 1e4a191366 Merge pull request #1284 from slayton58/pooler_end_logits_fp16_fix 
Fix fp16 masking in PoolerEndLogits
 2019-10-01 18:40:22 -04:00
Thomas Wolf ae50ad91ea Merge pull request #1362 from FeiWang96/doc 
fix link
 2019-09-28 10:26:42 +02:00
Thomas Wolf df7cd9e4e4 Merge pull request #1353 from wendingp/patch-1 
Fix some typos
 2019-09-27 23:00:34 +02:00
Thomas Wolf 6a17b3c51b Merge pull request #1355 from agrinh/master 
Fix tensorflow_dataset glue support
 2019-09-27 22:59:54 +02:00
Thomas Wolf 04e9a6f512 Merge pull request #1359 from dennymarcels/patch-1 
Update run_lm_finetuning.py
 2019-09-27 22:58:19 +02:00
Thomas Wolf ca559826c4 Merge pull request #1349 from ogabrielluiz/master 
Just some typos
 2019-09-27 13:08:00 +02:00
Thomas Wolf d83d295763 Merge pull request #1337 from mgrankin/fastdataset 
faster dataset building
 2019-09-27 10:35:12 +02:00
Thomas Wolf f6de000305 Merge pull request #1346 from BramVanroy/documentation 
Add small  note about the output of hidden states (closes #1332)
 2019-09-27 10:30:07 +02:00
Thomas Wolf 17ea43cf98 Merge pull request #1203 from huggingface/tf2 
[2.0] TF 2.0 support
 2019-09-26 12:11:03 +02:00
Thomas Wolf 4a233e5b2c Merge pull request #1315 from bryant1410/patch-1 
Remove unnecessary use of FusedLayerNorm
 2019-09-26 08:50:02 +02:00
Thomas Wolf e8e956dbb2 Merge pull request #1327 from huggingface/tf2-determinism 
Pytorch/TF2 determinism
 2019-09-24 22:49:57 +02:00
Thomas Wolf e4022d96f7 Merge pull request #1325 from huggingface/glue-included 
[Proposal] GLUE processors included in library
 2019-09-24 21:40:10 +02:00
Thomas Wolf 7c0f2d0a6a Merge pull request #1294 from sshleifer/delete-n-special-doc 
Delete n_special reference in docstring
 2019-09-23 14:54:55 +01:00
Thomas Wolf 0d1dad6d53 Merge pull request #1004 from erenup/master 
Refactoring old run_swag.py
 2019-09-18 21:42:51 +02:00
Thomas Wolf b62abe87c9 Merge pull request #1249 from ziliwang/master 
fixed: hard coding for max and min number will out of range in fp16, which will cause nan.
 2019-09-11 15:53:28 +02:00
Thomas Wolf 23c23f5399 Merge pull request #1229 from SKRohit/master 
changes in evaluate function in run_lm_finetuning.py
 2019-09-10 22:16:45 +02:00
Thomas Wolf 99a54ac51c Merge pull request #1233 from searchivarius/master 
Fix to prevent crashing on assert len(tokens_b)>=1
 2019-09-10 22:15:47 +02:00
Thomas Wolf 439b37b474 Merge pull request #1241 from mattolson93/patch-1 
Fixing typo in gpt2 for doc site's class link
 2019-09-10 22:14:18 +02:00
Thomas Wolf 2c177a87eb Merge pull request #1228 from huggingface/head-masking-test 
Trying to fix the head masking test
 2019-09-10 11:55:27 +02:00
Thomas Wolf 84d346b687 Merge pull request #1195 from huggingface/reorder_arguments 
[2.0] Reodering arguments for torch jit #1010 and future TF2.0 compatibility
 2019-09-09 15:42:51 +03:00
Thomas Wolf 3f05de6dde Merge branch 'master' into reorder_arguments 2019-09-09 15:42:25 +03:00
Thomas Wolf 995e38b7af Merge pull request #1214 from huggingface/new-examples 
Better examples
 2019-09-09 10:26:36 +03:00
Thomas Wolf 5ac8b62265 Merge pull request #1205 from maru0kun/patch-2 
Fix typo
 2019-09-05 21:44:16 +02:00
Thomas Wolf ed717635ff Merge pull request #1201 from huggingface/configuration_refactoring 
[2.0] - Split configuration and modeling files
 2019-09-05 21:16:58 +02:00
Thomas Wolf 9fd6e7ab9f Merge pull request #1190 from shijie-wu/xlm-tokenization 
Fix reference of import in XLM tokenization
 2019-09-04 12:50:49 +02:00
Thomas Wolf 0287d264e9 Merge pull request #1162 from huggingface/xlnet-bias 
XLNet bias fix on resize embeddings (cf #1124)
 2019-09-02 23:14:04 +02:00
Thomas Wolf 7b0c99add9 Merge pull request #1174 from huggingface/fix_byte_level_added_tokens 
Fix byte-level BPE decoding error when using added tokens
 2019-09-02 09:01:16 +02:00
Thomas Wolf b6cd856b08 Merge pull request #1164 from stefan-it/master 
distillation: fix ModuleNotFoundError error in token counts script
 2019-09-02 02:00:07 +02:00
Thomas Wolf ff7368eb6b Merge pull request #1077 from huggingface/pruning-save-and-load 
Pruning changes so that deleted heads are kept on save/load
 2019-09-01 09:42:15 +02:00
Thomas Wolf 51e980ce36 Merge pull request #1155 from anhnt170489/apex_fp16 
Update apex fp16 implementation
 2019-08-30 23:29:11 +02:00
Thomas Wolf 206c35e9a4 Merge pull request #1154 from ziliwang/master 
fix: hard coding for max number
 2019-08-30 23:23:08 +02:00
Thomas Wolf f3d18c71ec Merge pull request #1152 from epwalsh/fix-special-tokens 
fix adding special tokens
 2019-08-30 23:21:59 +02:00
Thomas Wolf d483cd8e46 Merge pull request #1074 from huggingface/improved_testing 
Shortcut to special tokens' ids - fix GPT2 & RoBERTa tokenizers - improved testing for GPT/GPT-2
 2019-08-30 23:18:58 +02:00
Thomas Wolf d2f21f08f5 Merge pull request #1092 from shijie-wu/xlm-tokenization 
Added cleaned configuration properties for tokenizer with serialization - improve tokenization of XLM
 2019-08-30 23:15:40 +02:00
Thomas Wolf 12b9cc9e26 Merge pull request #1110 from huggingface/automodels 
Torch.hub now based on AutoModels - Updating AutoModels with AutoModelWithLMHead, Sequence Classification and Question Answering
 2019-08-30 23:08:57 +02:00
Thomas Wolf cd65c41a83 Merge branch 'master' into xlm-tokenization 2019-08-30 17:15:16 +02:00
Thomas Wolf b66e9b4433 Merge pull request #1158 from rabeehk/master 
regarding #1026 pull request
 2019-08-30 16:30:33 +02:00
Thomas Wolf 0a2fecdf90 Merge branch 'master' into master 2019-08-30 16:30:08 +02:00
Thomas Wolf 41f35d0b3d Merge pull request #1089 from dhpollack/dhp/use_pytorch_layernorm 
change layernorm code to pytorch's native layer norm
 2019-08-30 14:49:08 +02:00
Thomas Wolf 01ad55f8cf Merge pull request #1026 from rabeehk/master 
loads the tokenizer for each checkpoint, to solve the reproducability…
 2019-08-30 14:15:36 +02:00
Thomas Wolf 50e615f43d Merge branch 'master' into improved_testing 2019-08-30 13:40:35 +02:00
Thomas Wolf f7978490b2 Merge pull request #1148 from huggingface/circleci 
Documentation auto-deploy
 2019-08-30 13:28:16 +02:00
Thomas Wolf 5f297c7be3 Merge pull request #1087 from huggingface/fix-warnings 
Decode now calls private property instead of public method
 2019-08-28 22:22:11 +02:00
Thomas Wolf d9847678b3 Merge pull request #1136 from adai183/update_SQuAD_script 
swap order of optimizer.step() and scheduler.step()
 2019-08-28 22:00:52 +02:00
Thomas Wolf 0f8ad89206 Merge pull request #1135 from stefan-it/master 
distilbert: fix number of hidden_size
 2019-08-28 22:00:12 +02:00
Thomas Wolf 0ecfd17f49 Merge pull request #987 from huggingface/generative-finetuning 
Generative finetuning
 2019-08-28 16:51:50 +02:00
Thomas Wolf 50792dbdcc Merge pull request #1127 from huggingface/dilbert 
DilBERT
 2019-08-28 16:43:09 +02:00
Thomas Wolf d06c5a2a0a Merge pull request #1120 from CrafterKolyan/patch-3 
Change attention mask dtype to be bool. Fix #1119
 2019-08-27 15:01:01 +02:00
Thomas Wolf edc5222fc3 Merge pull request #1118 from CrafterKolyan/patch-2 
Documentation fix #1117
 2019-08-27 14:58:50 +02:00
Thomas Wolf 9cf298dfc1 Merge pull request #1116 from CrafterKolyan/patch-1 
Delete nonexistent parameter from documentation fix #1115
 2019-08-27 14:56:43 +02:00
Thomas Wolf df9d6effae Merge pull request #1081 from huggingface/fix_distributed_barrier_hang 
Fix distributed barrier hang
 2019-08-23 16:53:53 +02:00
Thomas Wolf 3f20dd7186 Merge pull request #1075 from abhishekraok/modeling_utils_config_None 
reraise EnvironmentError in modeling_utils.py
 2019-08-23 12:42:39 +02:00
Thomas Wolf 90dcd8c05d Merge branch 'master' into generative-finetuning 2019-08-22 10:43:30 +02:00
Thomas Wolf 07681b6b58 Merge pull request #1064 from huggingface/gpt-2-large 
Adding gpt-2 large (774M parameters) model
 2019-08-21 03:05:56 +02:00