{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/duorc-towards-complex-language-understanding","title":"DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension","arxiv_id":"1804.07927","date":"2018-04-21","proceeding":"ACL 2018 7","authors":["Amrita Saha","Rahul Aralikatte","Mitesh M. Khapra","Karthik Sankaranarayanan"],"abstract":"We propose DuoRC, a novel dataset for Reading Comprehension (RC) that\nmotivates several new challenges for neural approaches in language\nunderstanding beyond those offered by existing RC datasets. DuoRC contains\n186,089 unique question-answer pairs created from a collection of 7680 pairs of\nmovie plots where each pair in the collection reflects two versions of the same\nmovie - one from Wikipedia and the other from IMDb - written by two different\nauthors. We asked crowdsourced workers to create questions from one version of\nthe plot and a different set of workers to extract or synthesize answers from\nthe other version. This unique characteristic of DuoRC where questions and\nanswers are created from different versions of a document narrating the same\nunderlying story, ensures by design, that there is very little lexical overlap\nbetween the questions created from one version and the segments containing the\nanswer in the other version. Further, since the two versions have different\nlevels of plot detail, narration style, vocabulary, etc., answering questions\nfrom the second version requires deeper language understanding and\nincorporating external background knowledge. Additionally, the narrative style\nof passages arising from movie plots (as opposed to typical descriptive\npassages in existing datasets) exhibits the need to perform complex reasoning\nover events across multiple sentences. Indeed, we observe that state-of-the-art\nneural RC models which have achieved near human performance on the SQuAD\ndataset, even when coupled with traditional NLP techniques to address the\nchallenges presented in DuoRC exhibit very poor performance (F1 score of 37.42%\non DuoRC v/s 86% on SQuAD dataset). This opens up several interesting research\navenues wherein DuoRC could complement other RC datasets to explore novel\nneural approaches for studying language understanding.","url_abs":"http://arxiv.org/abs/1804.07927v4","url_pdf":"http://arxiv.org/pdf/1804.07927v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"duorc-towards-complex-language-understanding","repo_url":"https://github.com/duorc/duorc","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"}],"methods":[],"datasets_introduced":[{"slug":"duorc","name":"DuoRC","full_name":"DuoRC"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1804.07927","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1804.07927"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/duorc/duorc","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"ran_honours":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f6c275d6a18330a9","entry":"exact_match_score","repo":"duorc/duorc","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/evaluate.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6c275d6a18330a9"}},{"code_sha256_prefix":"2112c433b9c6d343","entry":"f1_score","repo":"duorc/duorc","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/evaluate.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2112c433b9c6d343"}},{"code_sha256_prefix":"c6a80c065d2e4851","entry":"normalize_answer","repo":"duorc/duorc","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/evaluate.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6a80c065d2e4851"}},{"code_sha256_prefix":"68a0cecb1412d988","entry":"extract_entity_names","repo":"duorc/duorc","repo_kind":"official","path":"preprocessing/utils/nltk_ner.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/preprocessing/utils/nltk_ner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68a0cecb1412d988"}},{"code_sha256_prefix":"8562e36fb2d8669e","entry":"read_file_as_dict","repo":"duorc/duorc","repo_kind":"official","path":"preprocessing/utils/clean_utils.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/preprocessing/utils/clean_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8562e36fb2d8669e"}},{"code_sha256_prefix":"95c1595adbad8451","entry":"read_file_as_set","repo":"duorc/duorc","repo_kind":"official","path":"preprocessing/utils/clean_utils.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/preprocessing/utils/clean_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95c1595adbad8451"}},{"code_sha256_prefix":"c5c3fe94d2a210cb","entry":"text2int","repo":"duorc/duorc","repo_kind":"official","path":"preprocessing/utils/words2number.py","file_url":"https://github.com/duorc/duorc/blob/HEAD/preprocessing/utils/words2number.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5c3fe94d2a210cb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}