c-my/transformers
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
608 Commits 171 Branches 49 Tags
6a9038ba537e96c17bb166392f8f3c08a32719d3
Commit Graph
5 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Catalin Voss 4a49c22584 Warn instead of raising in BERT and GPT-2 tokenizers as well, to allow for pre-caching of tokens 2019-03-05 12:31:45 -08:00
Catalin Voss e99bc87e4d Merge branch 'patch-1' into patch-2 2019-03-05 12:24:18 -08:00
Catalin Voss 4b4b079272 Fix top k generation for k != 0 2019-03-02 21:54:44 -08:00
Catalin Voss 9775b2eb27 Allow tokenization of sequences > 512 for caching 
For many applications requiring randomized data access, it's easier to cache the tokenized representations than the words. So why not turn this into a warning?
 2019-03-02 16:30:21 -08:00
Catalin Voss c0cf0a04d5 Fix typo 2019-02-27 18:01:06 -08:00