</> Code with Papers by Syntology

Datasets › WMT 2014

WMT 2014

Introduced by Ondrej Bojar et al. in Findings of the 2014 Workshop on Statistical Machine Translation1 Jan 2014 archive 2025-07-28

WMT 2014 is a collection of datasets used in shared tasks of the Ninth Workshop on Statistical Machine Translation. The workshop featured four tasks:

  • a news translation task,
  • a quality estimation task,
  • a metrics task,
  • a medical text translation task.

Source: https://www.aclweb.org/anthology/W14-3302.pdf

HomepageBenchmarksPapers JSON

Benchmarks archive 2025-07-28

All 9 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

First row (archive order)PaperCode
Machine Translation WMT2014 English-German Transformer Cycle (Rev) BLEU score 35.14 Lessons on Parameter Sharing across Layers in Transformers takase/share_layer_params +1 91 Compare
Machine Translation WMT2014 English-French Transformer+BT (ADMIN init) BLEU score 46.4 Very Deep Transformers for Neural Machine Translation LiyuanLucasLiu/Transforemr-Clinic +3 57 Compare
Machine Translation WMT2014 German-English Bi-SimCut BLEU score 35.15 Bi-SimCut: A Simple Strategy for Boosting Neural Machine... gpengzhi/Bi-SimCut 16 Compare
Unsupervised Machine Translation WMT2014 French-English GPT-3 175B (Few-Shot) BLEU 39.2 Language Models are Few-Shot Learners ggml-org/llama.cpp +66 7 Compare
Unsupervised Machine Translation WMT2014 English-French BERT-fused NMT BLEU 38.27 Incorporating BERT into Neural Machine Translation bert-nmt/bert-nmt +2 7 Compare
Machine Translation WMT2014 French-English FLAN 137B (few-shot, k=9) BLEU score 37.9 Finetuned Language Models Are Zero-Shot Learners hiyouga/llama-efficient-tuning +7 3 Compare
Machine Translation WMT2014 English-Czech Evolved Transformer Big BLEU score 28.2 The Evolved Transformer tensorflow/tensor2tensor +2 2 Compare
Unsupervised Machine Translation WMT2014 English-German SMT + NMT (tuning and joint refinement) BLEU 22.5 An Effective Approach to Unsupervised Machine Translation artetxem/monoses 2 Compare
Unsupervised Machine Translation WMT2014 German-English SMT + NMT (tuning and joint refinement) BLEU 27.0 An Effective Approach to Unsupervised Machine Translation artetxem/monoses 2 Compare

Papers archive 2025-07-28

30 shown of 89 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 288. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.

DateSamples run Syntology
PartialFormer: Modeling Part Instead of Whole for Machine Translation 1 1 23 Oct 2023 not harvested
Mega: Moving Average Equipped Gated Attention 7 2 21 Sep 2022 ran 12 of 13 samples (1 unverified; 8 pointer-only for licence)
Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation 1 4 6 Jun 2022 not harvested
BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation 2 2 9 Sep 2021 ran 2 of 2 samples (0 unverified; 1 pointer-only for licence)
Finetuned Language Models Are Zero-Shot Learners 8 4 3 Sep 2021 ran 0 of 1 samples (1 unverified)
R-Drop: Regularized Dropout for Neural Networks 8 2 28 Jun 2021 ran 2 of 6 samples (4 unverified)
ResMLP: Feedforward networks for image classification with data-efficient training 19 4 7 May 2021 ran 2 of 7 samples (5 unverified)
Lessons on Parameter Sharing across Layers in Transformers 2 1 13 Apr 2021 ran 2 of 2 samples (0 unverified; 1 pointer-only for licence)
Rethinking Perturbations in Encoder-Decoders for Fast Training 1 1 5 Apr 2021 ran 1 of 1 samples (0 unverified; 1 pointer-only for licence)
Mask Attention Networks: Rethinking and Strengthen Transformer 1 2 25 Mar 2021 ran 0 of 3 samples (3 unverified)
Finetuning Pretrained Transformers into RNNs 2 2 24 Mar 2021 ran 3 of 8 samples (5 unverified; 4 pointer-only for licence)
Non-Autoregressive Translation by Learning Target Categorical Codes 1 2 21 Mar 2021 not harvested
Random Feature Attention 0 2 3 Mar 2021 not harvested
OmniNet: Omnidirectional Representations from Transformers 1 2 1 Mar 2021 not harvested
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks 1 1 5 Jan 2021 not harvested
Subformer: A Parameter Reduced Transformer 0 1 1 Jan 2021 not harvested
Incorporating a Local Translation Mechanism into Non-autoregressive Translation 1 4 12 Nov 2020 ran 3 of 5 samples (2 unverified)
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information 1 1 7 Oct 2020 not harvested
Very Deep Transformers for Neural Machine Translation 4 3 18 Aug 2020 ran 6 of 9 samples (3 unverified)
Glancing Transformer for Non-Autoregressive Neural Machine Translation 2 1 18 Aug 2020 not harvested
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation 0 3 21 Jun 2020 not harvested
Multi-branch Attentive Transformer 1 1 18 Jun 2020 ran 2 of 3 samples (1 unverified; 3 pointer-only for licence)
Language Models are Few-Shot Learners 67 2 28 May 2020 ran 15 of 65 samples (50 unverified; 4 pointer-only for licence)
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing 4 2 28 May 2020 not harvested
Synthesizer: Rethinking Self-Attention in Transformer Models 1 2 2 May 2020 ran 1 of 1 samples (0 unverified)
Lite Transformer with Long-Short Range Attention 2 2 24 Apr 2020 not harvested
Understanding the Difficulty of Training Transformers 2 1 17 Apr 2020 ran 4 of 5 samples (1 unverified)
PowerNorm: Rethinking Batch Normalization in Transformers 1 1 17 Mar 2020 not harvested
Learning to Encode Position for Transformer with Continuous Dynamical Model 1 2 13 Mar 2020 ran 3 of 6 samples (3 unverified; 6 pointer-only for licence)
Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule 5 1 9 Mar 2020 not harvested

The full list of 89 is in the JSON twin.

Dataset loaders archive 2025-07-28

huggingface.co/datasets/wmt/wmt14huggingface/datasetstfpytorchjax
huggingface.co/datasets/wmt14huggingface/datasetstfpytorchjax
parl.ai/docs/tasks.html#wmtfacebookresearch/ParlAIpytorch
www.tensorflow.org/datasets/catalog/wmt14_translatetensorflow/datasetstfjax

4 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

Machine TranslationTranslation deu-engTranslation eng-deuUnsupervised Machine Translation

License archive 2025-07-28

Unknown

Modalities archive 2025-07-28

Texts

Languages archive 2025-07-28

Tamil

Variants archive 2025-07-28

  • WMT-newstest2014-deen
  • WMT 2014 News
  • newstest2014-deen eng-deu
  • newstest2014-deen deu-eng
  • newstest2014-deen
  • wmt14
  • WMT 2014
  • WMT2014 German-English
  • WMT2014 English-Czech
  • WMT 2014 EN-FR
  • WMT 2014 EN-DE
  • WMT2014 English-German
  • WMT2014 French-English
  • WMT2014 English-French

14 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections