Julien ChaumondandSylvain Gugger 0cc5ab1333 Improve bert-japanese tokenizer handling (#8659) ...

* Make ci fail

* Try to make tests actually run?

* CI finally failing?

* Fix CI

* Revert "Fix CI"

This reverts commit ca7923be7334d4e571b023478ebdd6b33dfd0ebb.

* Ooops wrong one

* one more try

* Ok ok let's move this elsewhere

* Alternative to globals() (#8667)

* Alternative to globals()

* Error is raised later so return None

* Sentencepiece not installed make some tokenizers None

* Apply Lysandre wisdom

* Slightly clearer comment?

cc @sgugger

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

2020-11-23 11:15:02 -05:00

..

config.yml

Improve bert-japanese tokenizer handling (#8659)

2020-11-23 11:15:02 -05:00

deploy.sh

Update doc for v3.5.1

2020-11-13 10:29:58 -05:00