Papers › MultiFiT: Efficient Multi-lingual Language Model Fine-tuning

MultiFiT: Efficient Multi-lingual Language Model Fine-tuning

10 Sep 2019IJCNLP 2019 11arXiv:1909.04761archive 2025-07-28

Julian Martin Eisenschlos, Sebastian Ruder, Piotr Czapla, Marcin Kardas, Sylvain Gugger, Jeremy Howard

Pretrained language models are promising particularly for low-resource languages as they only require unlabelled data. However, training existing models requires huge amounts of compute, while pretrained cross-lingual models often underperform on low-resource languages. We propose Multi-lingual language model Fine-Tuning (MultiFiT) to enable practitioners to train and fine-tune language models efficiently in their own language. In addition, we propose a zero-shot method using an existing pretrained cross-lingual model. We evaluate our methods on two widely used cross-lingual classification datasets where they outperform models pretrained on orders of magnitude more data and compute. We release all models and code.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

TheophileBlard/french-sentiment-analysis-with-bert mentioned on GitHubtfMIT report

lukexyz/Language-Models mentioned on GitHubpytorch report

n-waves/multifit mentioned on GitHubMIT report

piegu/language-models mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Cross-Lingual Document Classification Document Classification Language Modeling Language Modelling model

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-Chinese	MultiFiT, pseudo	Accuracy	82.48	#2 of 5	Archive leaderboard	report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-French	MultiFiT, pseudo	Accuracy	89.42	#2 of 6	Archive leaderboard	report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-German	MultiFiT, pseudo	Accuracy	91.62%	#2 of 5	Archive leaderboard	report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-Italian	MultiFiT, pseudo	Accuracy	76.02	#1 of 4	Archive leaderboard	report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-Japanese	MultiFiT, pseudo	Accuracy	69.57	#1 of 3	Archive leaderboard	report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-Russian	MultiFiT, pseudo	Accuracy	67.83	#2 of 5	Archive leaderboard	report
Cross-Lingual Document Classification	MLDoc Zero-Shot English-to-Spanish	MultiFiT, pseudo	Accuracy	79.1	#2 of 6	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections