This commit is contained in:
jetrunner
2020-09-17 22:38:10 +08:00
parent 3cf1064932
commit 181b4c28a8
+5 -5
View File
@@ -1,20 +1,20 @@
## transformers.theseus
# transformers.theseus
![BERT of Theseus](https://github.com/JetRunner/BERT-of-Theseus/blob/master/bert-of-theseus.png?raw=true)
`transformers.theseus` is an implementation of [Theseus Compression](https://arxiv.org/abs/2002.02925).
Theseus compression exploits module replacing to compress a large model to a small one.
We implement [LayerDrop](https://arxiv.org/abs/1909.11556) and [Mixout](https://arxiv.org/abs/1909.11299) with the framework as well.
### Run BERT-of-Theseus
## Run BERT-of-Theseus
TBA.
### Run LayerDrop
## Run LayerDrop
TBA.
### Run Mixout
## Run Mixout
TBA.
### Citation
## Citation
Please consider citing this paper for the `theseus` framework and BERT-of-Theseus:
```bibtex
@misc{xu2020bertoftheseus,