Papers › On the Effectiveness of Compact Biomedical Transformers

On the Effectiveness of Compact Biomedical Transformers

7 Sep 2022arXiv:2209.03182archive 2025-07-28

Omid Rohanian, Mohammadmahdi Nouriborji, Samaneh Kouchaki, David A. Clifton

Language models pre-trained on biomedical corpora, such as BioBERT, have recently shown promising results on downstream biomedical tasks. Many existing pre-trained models, on the other hand, are resource-intensive and computationally heavy owing to factors such as embedding size, hidden dimension, and number of layers. The natural language processing (NLP) community has developed numerous strategies to compress these models utilising techniques such as pruning, quantisation, and knowledge distillation, resulting in models that are considerably faster, smaller, and subsequently easier to use in practice. By the same token, in this paper we introduce six lightweight models, namely, BioDistilBERT, BioTinyBERT, BioMobileBERT, DistilBioBERT, TinyBioBERT, and CompactBioBERT which are obtained either by knowledge distillation from a biomedical teacher or continual learning on the Pubmed dataset via the Masked Language Modelling (MLM) objective. We evaluate all of our models on three biomedical tasks and compare them with BioBERT-v1.1 to create efficient lightweight models that perform on par with their larger counterparts. All the models will be publicly available on our Huggingface profile at https://huggingface.co/nlpie and the codes used to run the experiments will be available at https://github.com/nlpie-research/Compact-Biomedical-Transformers.

PaperPDFCode

Code

nlpie-research/compact-biomedical-transformers officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Continual LearningKnowledge DistillationLanguage ModellingNamed Entity Recognition (NER)

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Named Entity Recognition (NER) BC2GM BioDistilBERT F1 86.97 #2 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC2GM CompactBioBERT F1 86.71 #3 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC2GM DistilBioBERT F1 86.6 #4 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC2GM BioMobileBERT F1 85.26 #7 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-chemical DistilBioBERT F1 94.53 #4 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-chemical BioDistilBERT F1 94.48 #5 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-chemical CompactBioBERT F1 94.31 #6 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-chemical BioMobileBERT F1 94.23 #7 of 13 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-disease BioDistilBERT F1 85.61 #7 of 10 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-disease DistilBioBERT F1 85.42 #8 of 10 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-disease CompactBioBERT F1 85.38 #9 of 10 Archive leaderboard report
Named Entity Recognition (NER) BC5CDR-disease BioMobileBERT F1 84.62 #10 of 10 Archive leaderboard report
Named Entity Recognition (NER) JNLPBA BioMobileBERT F1 80.13 #5 of 17 Archive leaderboard report
Named Entity Recognition (NER) JNLPBA DistilBioBERT F1 79.97 #7 of 17 Archive leaderboard report
Named Entity Recognition (NER) JNLPBA CompactBioBERT F1 79.88 #8 of 17 Archive leaderboard report
Named Entity Recognition (NER) JNLPBA BioDistilBERT F1 79.1 #10 of 17 Archive leaderboard report
Named Entity Recognition (NER) NCBI-disease CompactBioBERT F1 88.67 #10 of 26 Archive leaderboard report
Named Entity Recognition (NER) NCBI-disease DistilBioBERT F1 87.93 #14 of 26 Archive leaderboard report
Named Entity Recognition (NER) NCBI-disease BioDistilBERT F1 87.61 #18 of 26 Archive leaderboard report
Named Entity Recognition (NER) NCBI-disease BioMobileBERT F1 87.21 #20 of 26 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Knowledge Distillation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections