Papers › A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
Danqi Chen, Jason Bolton, Christopher D. Manning
Enabling a computer to understand a document so that it can answer comprehension questions is a central, yet unsolved goal of NLP. A key factor impeding its solution by machine learned systems is the limited availability of human-annotated data. Hermann et al. (2015) seek to solve this problem by creating over a million training examples by pairing CNN and Daily Mail news articles with their summarized bullet points, and show that a neural network can then be trained to give good performance on this task. In this paper, we conduct a thorough examination of this new reading comprehension task. Our primary aim is to understand what depth of language understanding is required to do well on this task. We approach this from one side by doing a careful hand-analysis of a small subset of the problems and from the other by showing that simple, carefully designed systems can obtain accuracies of 73.6% and 76.6% on these two datasets, exceeding current state-of-the-art results by 7-10% and approaching what we believe is the ceiling for performance on this task.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Question Answering | CNN / Daily Mail | Attentive + relabling + ensemble | CNN | 77.6 | #3 of 16 | Archive leaderboard | report |
| Question Answering | CNN / Daily Mail | Attentive + relabling + ensemble | Daily Mail | 79.2 | #3 of 16 | Archive leaderboard | report |
| Question Answering | CNN / Daily Mail | AttentiveReader + bilinear attention | CNN | 72.4 | #11 of 16 | Archive leaderboard | report |
| Question Answering | CNN / Daily Mail | AttentiveReader + bilinear attention | Daily Mail | 75.8 | #11 of 16 | Archive leaderboard | report |
| Question Answering | CNN / Daily Mail | Classifier | CNN | 67.9 | #14 of 16 | Archive leaderboard | report |
| Question Answering | CNN / Daily Mail | Classifier | Daily Mail | 68.3 | #14 of 16 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections