Papers › The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Niki Parmar, Mike Schuster, Zhifeng Chen, Yonghui Wu, Macduff Hughes
The past year has witnessed rapid advances in sequence-to-sequence (seq2seq) modeling for Machine Translation (MT). The classic RNN-based approaches to MT were first out-performed by the convolutional seq2seq model, which was then out-performed by the more recent Transformer model. Each of these new approaches consists of a fundamental architecture accompanied by a set of modeling and training techniques that are in principle applicable to other seq2seq architectures. In this paper, we tease apart the new architectures and their accompanying techniques in two ways. First, we identify several key modeling and training techniques, and apply them to the RNN architecture, yielding a new RNMT+ model that outperforms all of the three fundamental architectures on the benchmark WMT'14 English to French and English to German tasks. Second, we analyze the properties of each fundamental seq2seq architecture and devise new hybrid architectures intended to combine their strengths. Our hybrid models obtain further improvements, outperforming the RNMT+ model on both benchmark datasets.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Machine Translation | WMT2014 English-French | RNMT+ | BLEU score | 41.0 | #27 of 57 | Archive leaderboard | report |
| Machine Translation | WMT2014 English-French | RNMT+ | Hardware Burden | 132G | #27 of 57 | Archive leaderboard | report |
| Machine Translation | WMT2014 English-French | RNMT+ | Operations per network pass | 2.81G | #27 of 57 | Archive leaderboard | report |
| Machine Translation | WMT2014 English-German | RNMT+ | BLEU score | 28.5 | #41 of 91 | Archive leaderboard | report |
| Machine Translation | WMT2014 English-German | RNMT+ | Hardware Burden | 44G | #41 of 91 | Archive leaderboard | report |
| Machine Translation | WMT2014 English-German | RNMT+ | Operations per network pass | 2.81G | #41 of 91 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections