{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/torque-a-reading-comprehension-dataset-of","title":"TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions","arxiv_id":"2005.00242","date":"2020-05-01","proceeding":"EMNLP 2020 11","authors":["Qiang Ning","Hao Wu","Rujun Han","Nanyun Peng","Matt Gardner","Dan Roth"],"abstract":"A critical part of reading is being able to understand the temporal relationships between events described in a passage of text, even when those relationships are not explicitly stated. However, current machine reading comprehension benchmarks have practically no questions that test temporal phenomena, so systems trained on these benchmarks have no capacity to answer questions such as \"what happened before/after [some event]?\" We introduce TORQUE, a new English reading comprehension benchmark built on 3.2k news snippets with 21k human-generated questions querying temporal relationships. Results show that RoBERTa-large achieves an exact-match score of 51% on the test set of TORQUE, about 30% behind human performance.","url_abs":"https://arxiv.org/abs/2005.00242v2","url_pdf":"https://arxiv.org/pdf/2005.00242v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"machine-reading-comprehension","task_name":"Machine Reading Comprehension"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"}],"methods":[],"datasets_introduced":[{"slug":"torque","name":"Torque","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-torque","task":"Question Answering","dataset":"Torque","model":"RoBERTa-large","rank_in_archive_order":2,"of":2,"metrics":{"C":"34.5","EM":"51.1","F1":"75.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.00242","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}