Julien Chaumond
8a2d9bc9ef
* add empty model cards for every current DeepPavlov model * fix: replace cyrillic `с` with `c` * docs: add model cards for current DeepPavlov BERT models * docs: add links for arXiv preprints
language
| language | ||||
|---|---|---|---|---|
|
bert-base-bg-cs-pl-ru-cased
SlavicBERT[1] (Slavic (bg, cs, pl, ru), cased, 12-layer, 768-hidden, 12-heads, 180M parameters) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT.
[1]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. (2019). Tuning Multilingual Transformers for Language-Specific Named Entity Recognition. ACL anthology W19-3712.