{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-sentence-cloze-dataset-for-chinese-machine","title":"A Sentence Cloze Dataset for Chinese Machine Reading Comprehension","arxiv_id":"2004.03116","date":"2020-04-07","proceeding":"COLING 2020 8","authors":["Yiming Cui","Ting Liu","Ziqing Yang","Zhipeng Chen","Wentao Ma","Wanxiang Che","Shijin Wang","Guoping Hu"],"abstract":"Owing to the continuous efforts by the Chinese NLP community, more and more Chinese machine reading comprehension datasets become available. To add diversity in this area, in this paper, we propose a new task called Sentence Cloze-style Machine Reading Comprehension (SC-MRC). The proposed task aims to fill the right candidate sentence into the passage that has several blanks. We built a Chinese dataset called CMRC 2019 to evaluate the difficulty of the SC-MRC task. Moreover, to add more difficulties, we also made fake candidates that are similar to the correct ones, which requires the machine to judge their correctness in the context. The proposed dataset contains over 100K blanks (questions) within over 10K passages, which was originated from Chinese narrative stories. To evaluate the dataset, we implement several baseline systems based on the pre-trained models, and the results show that the state-of-the-art model still underperforms human performance by a large margin. We release the dataset and baseline system to further facilitate our community. Resources available through https://github.com/ymcui/cmrc2019","url_abs":"https://arxiv.org/abs/2004.03116v2","url_pdf":"https://arxiv.org/pdf/2004.03116v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-sentence-cloze-dataset-for-chinese-machine","repo_url":"https://github.com/ymcui/cmrc2019","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"machine-reading-comprehension","task_name":"Machine Reading Comprehension"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[{"slug":"cmrc-2019","name":"CMRC 2019","full_name":"Chinese Machine Reading Comprehension 2019"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2004.03116","atlas_url":"https://app.syntology.ai/?focus=2004.03116","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.03116"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ymcui/cmrc2019","reach":null}],"summary":{"ran_honours":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"9a5c75de52f06823","entry":"compare_list","repo":"ymcui/cmrc2019","repo_kind":"official","path":"eval/cmrc2019_evaluate.py","file_url":"https://github.com/ymcui/cmrc2019/blob/HEAD/eval/cmrc2019_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9a5c75de52f06823"}},{"code_sha256_prefix":"ba35d63536a11237","entry":"evaluate","repo":"ymcui/cmrc2019","repo_kind":"official","path":"eval/cmrc2019_evaluate.py","file_url":"https://github.com/ymcui/cmrc2019/blob/HEAD/eval/cmrc2019_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ba35d63536a11237"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}