0 Fork 0

Files

T

Julien Chaumond 8a2d9bc9ef Add model cards for DeepPavlov models (#3138 )

* add empty model cards for every current DeepPavlov model

* fix: replace cyrillic `с` with `c`

* docs: add model cards for current DeepPavlov BERT models

* docs: add links for arXiv preprints

2020-03-05 09:34:43 -05:00

README.md

Add model cards for DeepPavlov models (#3138 )

2020-03-05 09:34:43 -05:00

README.md

language

russian

rubert-base-cased-sentence

Sentence RuBERT (Russian, cased, 12-layer, 768-hidden, 12-heads, 180M parameters) is a representation-based sentence encoder for Russian. It is initialized with RuBERT and fine-tuned on SNLI[1] google-translated to russian and on russian part of XNLI dev set[2]. Sentence representations are mean pooled token embeddings in the same manner as in Sentence-BERT[3].

[1]: S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. (2015) A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326

[2]: Williams A., Bowman S. (2018) XNLI: Evaluating Cross-lingual Sentence Representations. arXiv preprint arXiv:1809.05053

[3]: N. Reimers, I. Gurevych (2019) Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084