Datasets › QNLI

QNLI (Question-answering NLI)

Introduced by Alex Wang et al. in GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding1 Jan 2019 archive 2025-07-28

The QNLI (Question-answering NLI) dataset is a Natural Language Inference dataset automatically derived from the Stanford Question Answering Dataset v1.1 (SQuAD). SQuAD v1.1 consists of question-paragraph pairs, where one of the sentences in the paragraph (drawn from Wikipedia) contains the answer to the corresponding question (written by an annotator). The dataset was converted into sentence pair classification by forming a pair between each question and each sentence in the corresponding context, and filtering out pairs with low lexical overlap between the question and the context sentence. The task is to determine whether the context sentence contains the answer to the question. This modified version of the original task removes the requirement that the model select the exact answer, but also removes the simplifying assumptions that the answer is always present in the input and that lexical overlap is a reliable cue. The QNLI dataset is part of GLUE benchmark.

Source: https://arxiv.org/pdf/1804.07461.pdf

Benchmarks archive 2025-07-28

All 3 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 22 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 1 placed): Natural Language Inference · QNLI (21 judged, 1 placed), Model Compression · QNLI (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

First row (archive order)PaperCode
Natural Language Inference QNLI ALBERT Accuracy 99.2% ALBERT: A Lite BERT for Self-supervised Learning of... huggingface/transformers +47 43 Compare
Data-free Knowledge Distillation QNLI GOLD (T5-base) Accuracy 91.7 GOLD: Generalized Knowledge Distillation via... — 4 Compare
Model Compression QNLI MobileBERT + 2bit-1dim model compression using DKM Accuracy 82.13 R2 Loss: Range Restriction Loss for Model Compression... — 2 Compare

Papers archive 2025-07-28

papers with a benchmark row: 37 · with a code link: 30 · where Syntology ran a sample: 22 (20 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (22 of 37 with a benchmark row: 20 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 37 of 37 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 1,234. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation 0 1 28 Mar 2024 not harvested
Prompt2Model: Generating Deployable Models from Natural Language Instructions 1 1 23 Aug 2023 not harvested
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning 1 1 29 May 2023 not harvested
R2 Loss: Range Restriction Loss for Model Compression and Quantization 0 2 14 Mar 2023 not harvested
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback 2 1 22 Oct 2022 not harvested
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale 4 1 15 Aug 2022 official: harvested, nothing ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
Adversarial Self-Attention for Language Understanding 1 2 25 Jun 2022 not harvested
ZeroGen: Efficient Zero-shot Learning via Dataset Generation 3 1 16 Feb 2022 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (7 pointer-only for licence)
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language 12 1 7 Feb 2022 community repositories only · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing 3 1 18 Nov 2021 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization 2 1 23 Jun 2021 community repositories only · 7 ran (of which 3 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified
FNet: Mixing Tokens with Fourier Transforms 12 1 9 May 2021 community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Entailment as Few-Shot Learner 3 1 29 Apr 2021 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
How to Train BERT with an Academic Budget 4 1 15 Apr 2021 not harvested
Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention 10 1 7 Feb 2021 official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence)
CLEAR: Contrastive Learning for Sentence Representation 0 1 31 Dec 2020 not harvested
RealFormer: Transformer Likes Residual Attention 5 1 21 Dec 2020 not harvested
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks 2 1 27 Oct 2020 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (1 pointer-only for licence)
Big Bird: Transformers for Longer Sequences 14 1 28 Jul 2020 official (archive's flag): 1 ran · 10 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified (11 pointer-only for licence)
SqueezeBERT: What can computer vision teach NLP about efficient neural networks? 6 1 19 Jun 2020 community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
DeBERTa: Decoding-enhanced BERT with Disentangled Attention 14 1 5 Jun 2020 official: no sample here; runs from other or unrecorded repositories · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified (3 pointer-only for licence)
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding 0 1 16 Mar 2020 not harvested
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization 6 4 8 Nov 2019 official: no sample here; runs from other or unrecorded repositories · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 1 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 57 5 23 Oct 2019 21 ran (of which 0 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 0 violated, 19 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified
Q8BERT: Quantized 8Bit BERT 5 1 14 Oct 2019 official (archive's flag): 5 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 2 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (3 pointer-only for licence)
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter 37 1 2 Oct 2019 official (archive's flag): 1 ran · 21 ran (of which 5 constructed an object rather than computing a result; 12 with no instrument failure: 3 honoured, 1 violated, 8 with no contract checked; 9 where Syntology's instrument failed) · 6 unverified (2 pointer-only for licence)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations 48 1 26 Sep 2019 official (archive's flag): 6 ran · 81 ran (of which 17 constructed an object rather than computing a result; 56 with no instrument failure: 4 honoured, 0 violated, 52 with no contract checked; 25 where Syntology's instrument failed) · 45 unverified (28 pointer-only for licence)
TinyBERT: Distilling BERT for Natural Language Understanding 10 2 23 Sep 2019 official: harvested, nothing ran · 0 ran · 4 unverified (4 pointer-only for licence)
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT 0 1 12 Sep 2019 not harvested
SenseBERT: Driving Some Sense into BERT 0 1 15 Aug 2019 not harvested
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding 0 1 13 Aug 2019 not harvested
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding 3 2 29 Jul 2019 official: harvested, nothing ran · 0 ran · 1 unverified (1 pointer-only for licence)
RoBERTa: A Robustly Optimized BERT Pretraining Approach 67 1 26 Jul 2019 community repositories only · 37 ran (of which 11 constructed an object rather than computing a result; 36 with no instrument failure: 2 honoured, 0 violated, 34 with no contract checked; 1 where Syntology's instrument failed) · 11 unverified (24 pointer-only for licence)
SpanBERT: Improving Pre-training by Representing and Predicting Spans 6 1 24 Jul 2019 official (archive's flag): 2 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 6 unverified (6 pointer-only for licence)
XLNet: Generalized Autoregressive Pretraining for Language Understanding 27 1 19 Jun 2019 official (archive's flag): 1 ran · 15 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 9 unverified (4 pointer-only for licence)
ERNIE: Enhanced Language Representation with Informative Entities 2 1 17 May 2019 official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 534 1 11 Oct 2018 official: no sample here; runs from other or unrecorded repositories · 300 ran (of which 75 constructed an object rather than computing a result; 234 with no instrument failure: 18 honoured, 5 violated, 211 with no contract checked; 66 where Syntology's instrument failed) · 359 unverified (164 pointer-only for licence)

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

CC BY-SA 4.0

Modalities archive 2025-07-28

Languages archive 2025-07-28

Variants archive 2025-07-28

  • QNLI
  • QNLI (8 training examples per class)
  • QNLI Dev

3 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections