Papers › Read + Verify: Machine Reading Comprehension with Unanswerable Questions
Read + Verify: Machine Reading Comprehension with Unanswerable Questions
Minghao Hu, Furu Wei, Yuxing Peng, Zhen Huang, Nan Yang, Dongsheng Li
Machine reading comprehension with unanswerable questions aims to abstain from answering when no answer can be inferred. In addition to extract answers, previous works usually predict an additional "no-answer" probability to detect unanswerable cases. However, they fail to validate the answerability of the question by verifying the legitimacy of the predicted answer. To address this problem, we propose a novel read-then-verify system, which not only utilizes a neural reader to extract candidate answers and produce no-answer probabilities, but also leverages an answer verifier to decide whether the predicted answer is entailed by the input snippets. Moreover, we introduce two auxiliary losses to help the reader better handle answer extraction as well as no-answer detection, and investigate three different architectures for the answer verifier. Our experiments on the SQuAD 2.0 dataset show that our system achieves a score of 74.2 F1 on the test set, achieving state-of-the-art results at the time of submission (Aug. 28th, 2018).
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Question Answering | SQuAD2.0 | Reinforced Mnemonic Reader + Answer Verifier (single model) | EM | 71.767 | #244 of 286 | Archive leaderboard | report |
| Question Answering | SQuAD2.0 | Reinforced Mnemonic Reader + Answer Verifier (single model) | F1 | 74.295 | #244 of 286 | Archive leaderboard | report |
| Question Answering | SQuAD2.0 dev | RMR + ELMo (Model-III) | EM | 72.3 | #11 of 13 | Archive leaderboard | report |
| Question Answering | SQuAD2.0 dev | RMR + ELMo (Model-III) | F1 | 74.8 | #11 of 13 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections