From b9c9e053811bc0bb3a76b3d495950585eac294ea Mon Sep 17 00:00:00 2001 From: sy-wada <62933006+sy-wada@users.noreply.github.com> Date: Fri, 15 May 2020 03:06:10 +0900 Subject: [PATCH] Create README.md (#4357) --- .../seiya/oubiobert-base-uncased/README.md | 39 +++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 model_cards/seiya/oubiobert-base-uncased/README.md diff --git a/model_cards/seiya/oubiobert-base-uncased/README.md b/model_cards/seiya/oubiobert-base-uncased/README.md new file mode 100644 index 00000000..9220aa08 --- /dev/null +++ b/model_cards/seiya/oubiobert-base-uncased/README.md @@ -0,0 +1,39 @@ +--- +tags: +- pytorch +- exbert +license: apache-2.0 +--- +# ouBioBERT-Base, Uncased +Bidirectional Encoder Representations from Transformers for Biomedical Text Mining by Osaka University (ouBioBERT) is a language model based on the BERT-Base (Devlin, et al., 2019) architecture. We pre-trained ouBioBERT on PubMed abstracts from the PubMed baseline (ftp://ftp.ncbi.nlm.nih.gov/pubmed/baseline) via our method. + +The details of the pre-training procedure can be found in Wada, et al. (2020). + +## Evaluation +We evaluated the performance of ouBioBERT in terms of the biomedical language understanding evaluation (BLUE) benchmark (Peng, et al., 2019). The numbers are mean (standard deviation) on five different random seeds. +| Dataset | Task Type | Score | +|:----------------|:--------------------------|-------------:| +| MedSTS | Sentence similarity | 84.9 (0.6) | +| BIOSSES | Sentence similarity | 92.3 (0.8) | +| BC5CDR-disease | Named-entity recognition | 87.4 (0.1) | +| BC5CDR-chemical | Named-entity recognition | 93.7 (0.2) | +| ShARe/CLEFE | Named-entity recognition | 80.1 (0.4) | +| DDI | Relation extraction | 81.1 (1.5) | +| ChemProt | Relation extraction | 75.0 (0.3) | +| i2b2 2010 | Relation extraction | 74.0 (0.8) | +| HoC | Document classification | 86.4 (0.5) | +| MedNLI | Inference | 83.6 (0.7) | +| **Total** | - |**83.8 (0.3)**| + +## Code for Fine-tuning +We made the source code for fine-tuning freely available at [our repository](https://github.com/sy-wada/blue_benchmark_with_transformers). + +## Citation +If you use our work in your research, please kindly cite the following paper: +```bibtex +now preparing... +``` + + + +