[Longformer] more models + model cards (#4628)
* adding freeze roberta models * model cards * lint
This commit is contained in:
1 parent
96f57c9ccb
commit
ef03ae874f
5 files changed
+67
-12
No files matched your search
@@ -0,0 +1,20 @@
|
||||
|
||||
# longformer-base-4096-extra.pos.embd.only
|
||||
|
||||
This model is similar to `longformer-base-4096` but it was pretrained to preserve RoBERTa weights by freezing all RoBERTa weights and only train the additional position embeddings.
|
||||
|
||||
|
||||
### Citing
|
||||
|
||||
If you use `Longformer` in your research, please cite [Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150).
|
||||
```
|
||||
@article{Beltagy2020Longformer,
|
||||
title={Longformer: The Long-Document Transformer},
|
||||
author={Iz Beltagy and Matthew E. Peters and Arman Cohan},
|
||||
journal={arXiv:2004.05150},
|
||||
year={2020},
|
||||
}
|
||||
```
|
||||
|
||||
`Longformer` is an open-source project developed by [the Allen Institute for Artificial Intelligence (AI2)](http://www.allenai.org).
|
||||
AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering.
|
||||
Reference in new issue
Block a user