Papers › Phrase-Based & Neural Unsupervised Machine Translation

Phrase-Based & Neural Unsupervised Machine Translation

20 Apr 2018EMNLP 2018 10arXiv:1804.07755archive 2025-07-28

Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato

Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs. This work investigates how to learn to translate when having access to only large monolingual corpora in each language. We propose two model variants, a neural and a phrase-based model. Both versions leverage a careful initialization of the parameters, the denoising effect of language models and automatic generation of parallel data by iterative back-translation. These models are significantly better than methods from the literature, while being simpler and having fewer hyper-parameters. On the widely used WMT'14 English-French and WMT'16 German-English benchmarks, our models respectively obtain 28.1 and 25.2 BLEU points without using a single parallel sentence, outperforming the state of the art by more than 11 BLEU points. On low-resource languages like English-Urdu and English-Romanian, our methods achieve even better results than semi-supervised and supervised approaches leveraging the paucity of available bitexts. Our code for NMT and PBSMT is publicly available.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

14 repositories listed; official and paper-mentioned ones first.

facebookresearch/UnsupervisedMT officialmentioned in papermentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
1-punchMan/CLTS mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Unsupervised Cross-lingual Representation Learning at Scale (arXiv:1911.02116; that paper's own run record: 40 ran (of which 13 constructed an object rather than computing a result; 35 with no instrument failure: 4 honoured, 1 violated, 30 with no contract checked; 5 where Syntology's instrument failed) · 19 unverified); nothing checks that they implement this paper's method. report
Helsinki-NLP/shared-info mentioned on GitHubSyntology: not harvested report
Kelechukwu1/PidginUNMT mentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
deterministic-algorithms-lab/Large-XLM mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Unsupervised Cross-lingual Representation Learning at Scale (arXiv:1911.02116; that paper's own run record: 40 ran (of which 13 constructed an object rather than computing a result; 35 with no instrument failure: 4 honoured, 1 violated, 30 with no contract checked; 5 where Syntology's instrument failed) · 19 unverified); nothing checks that they implement this paper's method. report
facebookresearch/UnsupervisedQA mentioned on GitHubNOASSERTIONSyntology: not harvested report
facebookresearch/XLM mentioned on GitHubpytorchSyntology: not harvested report
feyzaakyurek/XLM-LwLL mentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
fshdnc/enfi-XLM mentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
keleog/PidginUNMT mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Unsupervised Machine Translation Using Monolingual Corpora Only (arXiv:1711.00043; that paper's own run record: 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
kheeong/XLM_OWN mentioned on GitHubpytorchNOASSERTIONSyntology: not harvested report
luofuli/DualLanST mentioned on GitHubtfMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer (arXiv:1905.10060; that paper's own run record: 0 ran · 4 unverified); nothing checks that they implement this paper's method. report
luofuli/DualRL mentioned on GitHubtfMITSyntology: not harvested report
huggingface/transformers pytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Attention Is All You Need (arXiv:1706.03762; that paper's own run record: 610 ran (of which 293 constructed an object rather than computing a result; 529 with no instrument failure: 45 honoured, 15 violated, 469 with no contract checked; 81 where Syntology's instrument failed) · 336 unverified), HuggingFace's Transformers: State-of-the-art Natural Language Processing (arXiv:1910.03771; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified), DeBERTa: Decoding-enhanced BERT with Disentangled Attention (arXiv:2006.03654; that paper's own run record: 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified) and 1 more (every one in this page's JSON twin); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Machine TranslationNMTSentenceTranslationUnsupervised Machine Translation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Machine Translation WMT2014 English-French Unsupervised PBSMT BLEU score 28.11 #52 of 57 Archive leaderboard report
Machine Translation WMT2014 English-French PBSMT + NMT BLEU score 27.6 #53 of 57 Archive leaderboard report
Machine Translation WMT2014 English-French Unsupervised NMT + Transformer BLEU score 25.14 #56 of 57 Archive leaderboard report
Machine Translation WMT2014 English-German PBSMT + NMT BLEU score 20.23 #81 of 91 Archive leaderboard report
Machine Translation WMT2014 English-German Unsupervised PBSMT BLEU score 17.94 #85 of 91 Archive leaderboard report
Machine Translation WMT2014 English-German Unsupervised NMT + Transformer BLEU score 17.16 #87 of 91 Archive leaderboard report
Machine Translation WMT2016 English-Romanian PBSMT + NMT BLEU score 25.13 #16 of 21 Archive leaderboard report
Machine Translation WMT2016 English-Romanian Unsupervised PBSMT BLEU score 21.33 #17 of 21 Archive leaderboard report
Machine Translation WMT2016 English-Romanian Unsupervised NMT + Transformer BLEU score 21.18 #18 of 21 Archive leaderboard report
Machine Translation WMT2016 English-Russian PBSMT + NMT BLEU score 13.76 #2 of 4 Archive leaderboard report
Machine Translation WMT2016 English-Russian Unsupervised PBSMT BLEU score 13.37 #3 of 4 Archive leaderboard report
Machine Translation WMT2016 English-Russian Unsupervised NMT + Transformer BLEU score 7.98 #4 of 4 Archive leaderboard report
Unsupervised Machine Translation WMT2014 English-French PBSMT + NMT BLEU 27.6 #7 of 7 Archive leaderboard report
Unsupervised Machine Translation WMT2014 French-English PBSMT + NMT BLEU 27.7 #6 of 7 Archive leaderboard report
Unsupervised Machine Translation WMT2016 English-German PBSMT + NMT BLEU 20.2 #6 of 7 Archive leaderboard report
Unsupervised Machine Translation WMT2016 German-English PBSMT BLEU 25.2 #7 of 7 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections