Machine Reading Comprehension

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

4 leaderboard tables shown for this task, 4 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

43 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 43 until expanded.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 206 papers with code (555 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
  28 Nov 2016 14 repositories listed Syntology ran 3 of 33 samples · 30 unverified
  The size of the dataset and the fact that the questions are derived from real user search queries distinguishes MS MARCO from other well-known publicly available datasets for machine reading comprehension and…
• A Unified MRC Framework for Named Entity Recognition
  25 Oct 2019 8 repositories listed Syntology ran 2 of 5 samples · 3 unverified · 2 pointer-only (licence)
  Instead of treating the task of NER as a sequence labeling problem, we propose to formulate it as a machine reading comprehension (MRC) task.
• SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering
  10 Dec 2018 6 repositories listed
  Conversational question answering (CQA) is a novel QA task that requires understanding of dialogue context.
• Stochastic Answer Networks for Machine Reading Comprehension
  10 Dec 2017 6 repositories listed
  We propose a simple yet robust stochastic answer network (SAN) that simulates multi-step reasoning in machine reading comprehension.
• Stochastic Answer Networks for SQuAD 2.0
  24 Sep 2018 5 repositories listed
  This paper presents an extension of the Stochastic Answer Network (SAN), one of the state-of-the-art machine reading comprehension models, to be able to judge whether a question is unanswerable or not.
• Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
  18 Sep 2018 5 repositories listed
  We propose a multi-task learning framework to learn a joint Machine Reading Comprehension (MRC) model that can be applied to a wide range of MRC tasks in different domains.
• KLUE: Korean Language Understanding Evaluation
  20 May 2021 4 repositories listed
  We introduce Korean Language Understanding Evaluation (KLUE) benchmark.
• Dice Loss for Data-imbalanced NLP Tasks
  7 Nov 2019 4 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and the huge number of background examples…
• ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
  30 Jun 2021 3 repositories listed
  Recent pretraining models in Chinese neglect two important aspects specific to the Chinese language: glyph and pinyin, which carry significant syntax and semantic information for language understanding.
• DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications
  23 Apr 2020 3 repositories listed
  Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements.
• CLUE: A Chinese Language Understanding Evaluation Benchmark
  13 Apr 2020 3 repositories listed
  The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks.
• DUMA: Reading Comprehension with Transposition Thinking
  26 Jan 2020 3 repositories listed
  Multi-choice Machine Reading Comprehension (MRC) requires model to decide the correct answer from a set of answer options when given a passage and a question.
• DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
  14 Nov 2017 3 repositories listed
  Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements.
• Reinforced Mnemonic Reader for Machine Reading Comprehension
  8 May 2017 3 repositories listed
  In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects.
• The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants
  31 Aug 2023 2 repositories listed Syntology ran 0 of 6 samples · 6 unverified · 6 pointer-only (licence)
  We use this dataset to evaluate the capabilities of multilingual masked language models (MLMs) and large language models (LLMs).
• Rethinking Label Smoothing on Multi-hop Question Answering
  19 Dec 2022 2 repositories listed
  Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction.
• GENIUS: Sketch-based Language Model Pre-training via Extreme and Selective Masking for Text Generation and Augmentation
  18 Nov 2022 2 repositories listed
  We introduce GENIUS: a conditional text generation model using sketches as input, which can fill in the missing contexts for a given sketch (key information consisting of textual spans, phrases, or words, concatenated…
• Fact-driven Logical Reasoning for Machine Reading Comprehension
  21 May 2021 2 repositories listed
  Recent years have witnessed an increasing interest in training machines with reasoning ability, which deeply relies on accurately and clearly presented clue forms.
• Dependency Parsing as MRC-based Span-Span Prediction
  17 May 2021 2 repositories listed
  The proposed method has the following merits: (1) it addresses the fundamental problem that edges in a dependency tree should be constructed between subtrees; (2) the MRC framework allows the method to retrieve missing…
• Bidirectional Machine Reading Comprehension for Aspect Sentiment Triplet Extraction
  13 Mar 2021 2 repositories listed
  Aspect sentiment triplet extraction (ASTE), which aims to identify aspects from review sentences along with their corresponding opinion expressions and sentiments, is an emerging task in fine-grained opinion mining.
• ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning
  30 Dec 2020 2 repositories listed
  While pre-trained language models (PTLMs) have achieved noticeable success on many NLP tasks, they still struggle for tasks that require event temporal reasoning, which is essential for event-centric applications.
• LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
  16 Jul 2020 2 repositories listed
  Machine reading is a fundamental task for testing the capability of natural language understanding, which is closely related to human cognition in many aspects.
• Teaching Machine Comprehension with Compositional Explanations
  2 May 2020 2 repositories listed Syntology ran 10 of 10 samples · 0 unverified · 10 pointer-only (licence)
  Advances in machine reading comprehension (MRC) rely heavily on the collection of large scale human-annotated examples in the form of (question, paragraph, answer) triples.
• Retrospective Reader for Machine Reading Comprehension
  27 Jan 2020 2 repositories listed
  Inspired by how humans solve reading comprehension questions, we proposed a retrospective reader (Retro-Reader) that integrates two stages of reading and verification strategies: 1) sketchy reading that briefly…
• Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus
  27 Jan 2020 2 repositories listed Syntology ran 1 of 8 samples · 7 unverified · 2 pointer-only (licence)
  In this paper, we propose Answer-Clue-Style-aware Question Generation (ACS-QG), which aims at automatically generating high-quality and diverse question-answer pairs from unlabeled text corpus at scale by imitating the…
• NumNet: Machine Reading Comprehension with Numerical Reasoning
  15 Oct 2019 2 repositories listed
  Numerical reasoning, such as addition, subtraction, sorting and counting is a critical skill in human's reading comprehension, which has not been well considered in existing machine reading comprehension (MRC) systems.
• MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension
  1 Oct 2019 2 repositories listed
  Machine Reading Comprehension (MRC) for question answering (QA), which aims to answer a question given the relevant context passages, is an important way to test the ability of intelligence systems to understand human…
• Pre-Training with Whole Word Masking for Chinese BERT
  19 Jun 2019 2 repositories listed
  To demonstrate the effectiveness of these models, we create a series of Chinese pre-trained language models as our baselines, including BERT, RoBERTa, ELECTRA, RBT, etc.
• Text Understanding with the Attention Sum Reader Network
  4 Mar 2016 2 repositories listed Syntology ran 0 of 10 samples · 10 unverified
  Several large cloze-style context-question-answer datasets have been introduced recently: the CNN and Daily Mail news data and the Children's Book Test.
• MRCEval: A Comprehensive, Challenging and Accessible Machine Reading Comprehension Benchmark
  10 Mar 2025 1 repository listed
  Existing MRC datasets primarily assess specific aspects of reading comprehension (RC), lacking a comprehensive MRC benchmark.

Syntology lines on 7 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.