Update tokenization_t5.py (#5717)

Minor doc fix.
This commit is contained in:
Gaurav Mishra
2020-07-14 00:02:03 -04:00
committed by GitHub
parent c3c61ea017
commit f0bda06f43
+1 -1
View File
@@ -61,7 +61,7 @@ PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES = {
class T5Tokenizer(PreTrainedTokenizer):
"""
Constructs an XLNet tokenizer. Based on `SentencePiece <https://github.com/google/sentencepiece>`__ .
Constructs a T5 tokenizer. Based on `SentencePiece <https://github.com/google/sentencepiece>`__ .
This tokenizer inherits from :class:`~transformers.PreTrainedTokenizer` which contains most of the methods. Users
should refer to the superclass for more information regarding methods.