Julien Chaumond
8a2d9bc9ef
* add empty model cards for every current DeepPavlov model * fix: replace cyrillic `с` with `c` * docs: add model cards for current DeepPavlov BERT models * docs: add links for arXiv preprints
533 B
533 B
language
| language | |
|---|---|
|
rubert-base-cased
RuBERT (Russian, cased, 12-layer, 768-hidden, 12-heads, 180M parameters) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT-base as an initialization for RuBERT[1].
[1]: Kuratov, Y., Arkhipov, M. (2019). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint arXiv:1905.07213.