Datasets › SQuAD

SQuAD (Stanford Question Answering Dataset)

Introduced by François Bienvenu et al. in 0-1 laws for pattern occurrences in phylogenetic trees and networks7 Feb 2024 archive 2025-07-28

The Stanford Question Answering Dataset (SQuAD) is a collection of question-answer pairs derived from Wikipedia articles. In SQuAD, the correct answers of questions can be any sequence of tokens in the given text. Because the questions and answers are produced by humans through crowdsourcing, it is more diverse than some other question-answering datasets. SQuAD 1.1 contains 107,785 question-answer pairs on 536 articles. SQuAD2.0 (open-domain SQuAD, SQuAD-Open), the latest version, combines the 100,000 questions in SQuAD1.1 with over 50,000 un-answerable questions written adversarially by crowdworkers in forms that are similar to the answerable ones.

Source: Deep Learning Based Text Classification: A Comprehensive Review Image Source: https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/Prime_number.html

Benchmarks archive 2025-07-28

All 12 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 3 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): Question Answering · SQuAD1.1 (3 judged, 0 placed), Question Answering · SQuAD1.1 dev (2 judged, 0 placed), Open-Domain Question Answering · SQuAD1.1 dev (1 judged, 0 placed), Open-Domain Question Answering · SQuAD1.1 (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

Papers archive 2025-07-28

papers with a benchmark row: 82 · with a code link: 63 · where Syntology ran a sample: 32 (29 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (32 of 82 with a benchmark row: 29 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 82 of 82 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 151. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation 0 1 28 Mar 2024 not harvested
Blended RAG: Improving RAG (Retriever-Augmented Generation) Accuracy with Semantic Search and Hybrid Query-Based Retrievers 1 2 22 Mar 2024 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Prompt2Model: Generating Deployable Models from Natural Language Instructions 1 1 23 Aug 2023 not harvested
TextBox 2.0: A Text Generation Library with Pre-trained Language Models 1 2 26 Dec 2022 official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result
DyREx: Dynamic Query Representation for Extractive Question Answering 1 1 26 Oct 2022 official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (3 pointer-only for licence)
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback 2 1 22 Oct 2022 not harvested
Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation 0 1 9 Sep 2022 not harvested
LinkBERT: Pretraining Language Models with Document Links 1 1 29 Mar 2022 official: harvested, nothing ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 11 unverified
ZeroGen: Efficient Zero-shot Learning via Dataset Generation 3 1 16 Feb 2022 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (7 pointer-only for licence)
Information Theoretic Representation Distillation 1 2 1 Dec 2021 not harvested
Prune Once for All: Sparse Pre-Trained Language Models 2 9 10 Nov 2021 not harvested
Ensemble ALBERT on SQuAD 2.0 1 1 19 Oct 2021 not harvested
Fine-tune the Entire RAG Architecture (including DPR retriever) for Question-Answering 2 1 22 Jun 2021 not harvested
Pay Attention to MLPs 20 1 17 May 2021 34 ran (of which 15 constructed an object rather than computing a result; 31 with no instrument failure: 1 honoured, 4 violated, 26 with no contract checked; 3 where Syntology's instrument failed) · 10 unverified (11 pointer-only for licence)
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes 0 1 12 Feb 2021 not harvested
EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System 0 1 6 Jan 2021 not harvested
Learning to Generate Questions by Recovering Answer-containing Sentences 0 2 1 Jan 2021 not harvested
Learning Dense Representations of Phrases at Scale 4 1 23 Dec 2020 not harvested
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention 9 7 2 Oct 2020 official (archive's flag): 3 ran · 3 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 7 unverified
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval 1 1 28 Sep 2020 not harvested
DeBERTa: Decoding-enhanced BERT with Disentangled Attention 14 1 5 Jun 2020 official: no sample here; runs from other or unrecorded repositories · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified (3 pointer-only for licence)
Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs 1 2 28 May 2020 official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (3 pointer-only for licence)
Harvesting and Refining Question-Answer Pairs for Unsupervised QA 1 2 6 May 2020 not harvested
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training 3 1 28 Feb 2020 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified
Retrospective Reader for Machine Reading Comprehension 2 4 27 Jan 2020 not harvested
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation 5 1 26 Jan 2020 not harvested
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training 5 1 13 Jan 2020 not harvested
Dice Loss for Data-imbalanced NLP Tasks 4 2 7 Nov 2019 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
A Recurrent BERT-based Model for Question Generation 1 1 1 Nov 2019 not harvested
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension 47 1 29 Oct 2019 40 ran (of which 7 constructed an object rather than computing a result; 30 with no instrument failure: 1 honoured, 2 violated, 27 with no contract checked; 10 where Syntology's instrument failed) · 13 unverified (13 pointer-only for licence)
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 57 5 23 Oct 2019 21 ran (of which 0 constructed an object rather than computing a result; 20 with no instrument failure: 1 honoured, 0 violated, 19 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter 37 2 2 Oct 2019 official (archive's flag): 1 ran · 21 ran (of which 5 constructed an object rather than computing a result; 12 with no instrument failure: 3 honoured, 1 violated, 8 with no contract checked; 9 where Syntology's instrument failed) · 6 unverified (2 pointer-only for licence)
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations 48 6 26 Sep 2019 official (archive's flag): 6 ran · 81 ran (of which 17 constructed an object rather than computing a result; 56 with no instrument failure: 4 honoured, 0 violated, 52 with no contract checked; 25 where Syntology's instrument failed) · 45 unverified (28 pointer-only for licence)
TinyBERT: Distilling BERT for Natural Language Understanding 10 2 23 Sep 2019 official: harvested, nothing ran · 0 ran · 4 unverified (4 pointer-only for licence)
Semantics-aware BERT for Language Understanding 1 4 5 Sep 2019 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (3 pointer-only for licence)
Mixture Content Selection for Diverse Sequence Generation 1 1 4 Sep 2019 not harvested
SG-Net: Syntax-Guided Machine Reading Comprehension 1 5 14 Aug 2019 not harvested
RoBERTa: A Robustly Optimized BERT Pretraining Approach 67 2 26 Jul 2019 community repositories only · 37 ran (of which 11 constructed an object rather than computing a result; 36 with no instrument failure: 2 honoured, 0 violated, 34 with no contract checked; 1 where Syntology's instrument failed) · 11 unverified (24 pointer-only for licence)
SpanBERT: Improving Pre-training by Representing and Predicting Spans 6 3 24 Jul 2019 official (archive's flag): 2 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 6 unverified (6 pointer-only for licence)
XLNet: Generalized Autoregressive Pretraining for Language Understanding 27 4 19 Jun 2019 official (archive's flag): 1 ran · 15 ran (of which 2 constructed an object rather than computing a result; 10 with no instrument failure: 2 honoured, 0 violated, 8 with no contract checked; 5 where Syntology's instrument failed) · 9 unverified (4 pointer-only for licence)
Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language 2 1 17 May 2019 not harvested
Unified Language Model Pre-training for Natural Language Understanding and Generation 9 1 8 May 2019 not harvested
Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering 0 1 14 Apr 2019 not harvested
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes 32 1 1 Apr 2019 community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (9 pointer-only for licence)
Evaluating Rewards for Question Generation Models 1 1 28 Feb 2019 not harvested
End-to-End Open-Domain Question Answering with BERTserini 1 1 5 Feb 2019 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
A Fully Attention-Based Information Retriever 1 2 22 Oct 2018 not harvested
U-Net: Machine Reading Comprehension with Unanswerable Questions 1 2 12 Oct 2018 not harvested
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 534 6 11 Oct 2018 official: no sample here; runs from other or unrecorded repositories · 300 ran (of which 75 constructed an object rather than computing a result; 234 with no instrument failure: 18 honoured, 5 violated, 211 with no contract checked; 66 where Syntology's instrument failed) · 359 unverified (164 pointer-only for licence)
Explicit Utilization of General Knowledge in Machine Reading Comprehension 0 2 10 Sep 2018 not harvested
Read + Verify: Machine Reading Comprehension with Unanswerable Questions 0 2 17 Aug 2018 not harvested
A Multi-Stage Memory Augmented Neural Network for Machine Reading Comprehension 0 1 1 Jul 2018 not harvested
Leveraging Context Information for Natural Question Generation 1 1 1 Jun 2018 not harvested
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension 15 4 23 Apr 2018 14 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 3 honoured, 0 violated, 7 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (3 pointer-only for licence)
Deep contextualized word representations 46 4 15 Feb 2018 30 ran (of which 12 constructed an object rather than computing a result; 24 with no instrument failure: 2 honoured, 0 violated, 22 with no contract checked; 6 where Syntology's instrument failed) · 28 unverified (25 pointer-only for licence)
Contextualized Word Representations for Reading Comprehension 1 2 10 Dec 2017 not harvested
Stochastic Answer Networks for Machine Reading Comprehension 6 5 10 Dec 2017 not harvested
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension 3 4 16 Nov 2017 not harvested
DCN+: Mixed Objective and Deep Residual Coattention for Question Answering 1 3 31 Oct 2017 not harvested
Simple and Effective Multi-Paragraph Reading Comprehension 1 1 29 Oct 2017 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified
Phase Conductor on Multi-layered Attentions for Machine Comprehension 0 4 28 Oct 2017 not harvested
Smarnet: Teaching Machines to Read and Comprehend Like Human 0 2 8 Oct 2017 not harvested
Simple Recurrent Units for Highly Parallelizable Recurrence 11 2 8 Sep 2017 community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
Learned in Translation: Contextualized Word Vectors 5 2 1 Aug 2017 community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension 0 3 28 Jul 2017 not harvested
Gated Self-Matching Networks for Reading Comprehension and Question Answering 0 2 1 Jul 2017 not harvested
Learning to Compute Word Embeddings On the Fly 0 4 1 Jun 2017 not harvested
Reinforced Mnemonic Reader for Machine Reading Comprehension 3 5 8 May 2017 not harvested
Ruminating Reader: Reasoning with Gated Multi-Hop Attention 0 2 24 Apr 2017 not harvested
Neural Question Generation from Text: A Preliminary Study 6 1 6 Apr 2017 community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Reading Wikipedia to Answer Open-Domain Questions 10 3 31 Mar 2017 community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering 0 3 14 Mar 2017 not harvested
Making Neural QA as Simple as Possible but not Simpler 3 3 14 Mar 2017 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified
Structural Embedding of Syntactic Trees for Machine Comprehension 0 6 2 Mar 2017 not harvested
Multi-Perspective Context Matching for Machine Comprehension 1 4 13 Dec 2016 not harvested
Words or Characters? Fine-grained Gating for Reading Comprehension 1 2 6 Nov 2016 not harvested
Bidirectional Attention Flow for Machine Comprehension 27 3 5 Nov 2016 official (archive's flag): 1 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 2 violated, 1 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (7 pointer-only for licence)
Dynamic Coattention Networks For Question Answering 6 4 5 Nov 2016 not harvested
Learning Recurrent Span Representations for Extractive Question Answering 2 2 4 Nov 2016 not harvested
End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension 0 2 31 Oct 2016 not harvested
ReasoNet: Learning to Stop Reading in Machine Comprehension 0 2 17 Sep 2016 not harvested
Machine Comprehension Using Match-LSTM and Answer Pointer 5 5 29 Aug 2016 official: no sample here; runs from other or unrecorded repositories · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)

Dataset loaders archive 2025-07-28

38 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

CC BY-SA 4.0

Modalities archive 2025-07-28

Languages archive 2025-07-28

Variants archive 2025-07-28

  • squad_bn
  • squad_adversarial
  • qg_squad
  • The Stanford Question Answering Dataset
  • squad_v2
  • SQuAD
  • SQuAD2.0 dev
  • SQuAD2.0
  • SQuAD1.1 dev
  • SQuAD1.1

10 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections