Reading Comprehension

Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.

Benchmarks archive 2025-07-28

7 leaderboard tables shown for this task, 7 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

95 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 95 until expanded.

Subtasks archive 2025-07-28

20 subtasks in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 634 papers with code (1,760 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Language Models are Few-Shot Learners
  28 May 2020 67 repositories listed Syntology ran 15 of 65 samples · 50 unverified · 4 pointer-only (licence)
  By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
  26 Jul 2019 67 repositories listed Syntology ran 22 of 48 samples · 26 unverified · 23 pointer-only (licence)
  Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
• Listen, Attend and Spell
  5 Aug 2015 40 repositories listed Syntology ran 12 of 51 samples · 39 unverified · 9 pointer-only (licence)
  Unlike traditional DNN-HMM models, this model learns all the components of a speech recognizer jointly.
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
  19 Jun 2019 27 repositories listed Syntology ran 10 of 24 samples · 14 unverified · 3 pointer-only (licence)
  With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling.
• Bidirectional Attention Flow for Machine Comprehension
  5 Nov 2016 27 repositories listed Syntology ran 8 of 11 samples · 3 unverified · 7 pointer-only (licence)
  Machine comprehension (MC), answering a query about a given context paragraph, requires modeling complex interactions between the context and the query.
• Language Models are Unsupervised Multitask Learners
  14 Feb 2019 21 repositories listed
  Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically approached with supervised learning on taskspecific datasets.
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
  16 Jun 2016 21 repositories listed Syntology ran 4 of 6 samples · 2 unverified · 6 pointer-only (licence)
  We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100, 000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question…
• Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
  19 Feb 2015 20 repositories listed Syntology ran 1 of 3 samples · 2 unverified · 2 pointer-only (licence)
  One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.
• QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
  23 Apr 2018 15 repositories listed Syntology ran 7 of 19 samples · 12 unverified · 2 pointer-only (licence)
  On the SQuAD dataset, our model is 3x to 13x faster in training and 4x to 9x faster in inference, while achieving equivalent accuracy to recurrent models.
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  5 Jun 2020 14 repositories listed Syntology ran 4 of 13 samples · 9 unverified · 3 pointer-only (licence)
  Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
  28 Nov 2016 14 repositories listed Syntology ran 3 of 33 samples · 30 unverified
  The size of the dataset and the fact that the questions are derived from real user search queries distinguishes MS MARCO from other well-known publicly available datasets for machine reading comprehension and…
• Teaching Machines to Read and Comprehend
  10 Jun 2015 12 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  Teaching machines to read natural language documents remains an elusive challenge.
• Know What You Don't Know: Unanswerable Questions for SQuAD
  11 Jun 2018 11 repositories listed
  Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated in…
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
  17 Sep 2019 10 repositories listed Syntology ran 12 of 47 samples · 35 unverified · 15 pointer-only (licence)
  To demonstrate that large language models can further advance the state of the art (SOTA), we train an 8.
• Learning to Ask: Neural Question Generation for Reading Comprehension
  29 Apr 2017 10 repositories listed
  We study automatic question generation for sentences from text passages in reading comprehension.
• Reading Wikipedia to Answer Open-Domain Questions
  31 Mar 2017 10 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  This paper proposes to tackle open- domain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia article.
• mT5: A massively multilingual pre-trained text-to-text transformer
  22 Oct 2020 8 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks.
• A Unified MRC Framework for Named Entity Recognition
  25 Oct 2019 8 repositories listed Syntology ran 2 of 5 samples · 3 unverified · 2 pointer-only (licence)
  Instead of treating the task of NER as a sequence labeling problem, we propose to formulate it as a machine reading comprehension (MRC) task.
• PaLM: Scaling Language Modeling with Pathways
  5 Apr 2022 7 repositories listed Syntology ran 30 of 37 samples · 7 unverified
  To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.
• Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
  10 Nov 2019 7 repositories listed Syntology ran 4 of 6 samples · 2 unverified · 4 pointer-only (licence)
  We introduce an approach for open-domain question answering (QA) that retrieves and reads a passage graph, where vertices are passages of text and edges represent relationships that are derived from an external…
• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
  10 Dec 2018 6 repositories listed
  Conversational question answering (CQA) is a novel QA task that requires understanding of dialogue context.
• Stochastic Answer Networks for Machine Reading Comprehension
  10 Dec 2017 6 repositories listed
  We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension.
• PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
  26 Apr 2021 5 repositories listed
  To enhance the generalization ability of PanGu-α, we collect 1.
• Stochastic Answer Networks for SQuAD 2.0
  24 Sep 2018 5 repositories listed
  This paper presents an extension of the Stochastic Answer Network (SAN), one of the state-of-the-art machine reading comprehension models, to be able to judge whether a question is unanswerable or not.
• Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
  18 Sep 2018 5 repositories listed
  We propose a multi-task learning framework to learn a joint Machine Reading Comprehension (MRC) model that can be applied to a wide range of MRC tasks in different domains.
• Machine Comprehension Using Match-LSTM and Answer Pointer
  29 Aug 2016 5 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  We propose two ways of using Pointer Net for our task.
• KLUE: Korean Language Understanding Evaluation
  20 May 2021 4 repositories listed
  We introduce Korean Language Understanding Evaluation (KLUE) benchmark.
• Learning Dense Representations of Phrases at Scale
  23 Dec 2020 4 repositories listed
  Open-domain question answering can be reformulated as a phrase retrieval problem, without the need for processing documents on-demand during inference (Seo et al., 2019).
• Break It Down: A Question Understanding Benchmark
  31 Jan 2020 4 repositories listed
  Understanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer.
• Dice Loss for Data-imbalanced NLP Tasks
  7 Nov 2019 4 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and the huge number of background examples…

Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.