Fixed all links. Removed TPU. Changed CLI to Converting TF models. Many minor formatting adjustments. Added "TODO Lysandre filled" where necessary.

This commit is contained in:
LysandreJik committed 2019-07-10 14:45:56 -04:00
1 parent 3f56ad5aff
commit f773faa258
19 files changed
+235 -153

No files matched your search

+2 -1
View File
@@ -52,7 +52,8 @@ SEG_ID_PAD = 4
class XLNetTokenizer(PreTrainedTokenizer):
"""
SentencePiece based tokenizer. Peculiarities:
- requires SentencePiece: https://github.com/google/sentencepiece
- requires `SentencePiece <https://github.com/google/sentencepiece>`_
"""
vocab_files_names = VOCAB_FILES_NAMES
pretrained_vocab_files_map = PRETRAINED_VOCAB_FILES_MAP