Commit Graph
3 Commits
Author SHA1 Message Date
Sylvain Gugger 37be3786cf Clean documentation (#4849)
* Clean documentation
2020-06-08 11:28:19 -04:00
Sam Shleifer 9a687ebb77 [Marian Fixes] prevent predicting pad_token_id before softmax, support language codes, name multilingual models (#4290) 2020-05-13 17:29:41 -04:00
Sam Shleifer 3487be75ef [Marian] documentation and AutoModel support (#4152)
- MarianSentencepieceTokenizer - > MarianTokenizer
- Start using unk token.
- add docs page
- add better generation params to MarianConfig
- more conversion utilities
2020-05-10 13:54:57 -04:00