{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/achieving-human-parity-on-automatic-chinese","title":"Achieving Human Parity on Automatic Chinese to English News Translation","arxiv_id":"1803.05567","date":"2018-03-15","proceeding":null,"authors":["Hany Hassan","Anthony Aue","Chang Chen","Vishal Chowdhary","Jonathan Clark","Christian Federmann","Xuedong Huang","Marcin Junczys-Dowmunt","William Lewis","Mu Li","Shujie Liu","Tie-Yan Liu","Renqian Luo","Arul Menezes","Tao Qin","Frank Seide","Xu Tan","Fei Tian","Lijun Wu","Shuangzhi Wu","Yingce Xia","Dong-dong Zhang","Zhirui Zhang","Ming Zhou"],"abstract":"Machine translation has made rapid advances in recent years. Millions of\npeople are using it today in online translation systems and mobile applications\nin order to communicate across language barriers. The question naturally arises\nwhether such systems can approach or achieve parity with human translations. In\nthis paper, we first address the problem of how to define and accurately\nmeasure human parity in translation. We then describe Microsoft's machine\ntranslation system and measure the quality of its translations on the widely\nused WMT 2017 news translation task from Chinese to English. We find that our\nlatest neural machine translation system has reached a new state-of-the-art,\nand that the translation quality is at human parity when compared to\nprofessional human translations. We also find that it significantly exceeds the\nquality of crowd-sourced non-professional translations.","url_abs":"http://arxiv.org/abs/1803.05567v2","url_pdf":"http://arxiv.org/pdf/1803.05567v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"achieving-human-parity-on-automatic-chinese","repo_url":"https://github.com/cfedermann/Appraise","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"achieving-human-parity-on-automatic-chinese","repo_url":"https://github.com/sanxing-chen/NMT2017-ZH-EN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/machine-translation-on-wmt-2017-english-1","task":"Machine Translation","dataset":"WMT 2017 English-Chinese","model":"Hassan et al. (2018)","rank_in_archive_order":3,"of":3,"metrics":{"BLEU score":"24.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.05567","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1803.05567"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cfedermann/Appraise","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sanxing-chen/NMT2017-ZH-EN","reach":null}],"summary":{"ran_violates":2,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"c6850177d946f671","entry":"is_lang","repo":"sanxing-chen/NMT2017-ZH-EN","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/sanxing-chen/NMT2017-ZH-EN/blob/HEAD/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c6850177d946f671"}},{"code_sha256_prefix":"9708561102ba41d7","entry":"is_sgm","repo":"sanxing-chen/NMT2017-ZH-EN","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/sanxing-chen/NMT2017-ZH-EN/blob/HEAD/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9708561102ba41d7"}},{"code_sha256_prefix":"3ca4d1464b186e3d","entry":"preprocess_text","repo":"sanxing-chen/NMT2017-ZH-EN","repo_kind":"listed","path":"preprocess.py","file_url":"https://github.com/sanxing-chen/NMT2017-ZH-EN/blob/HEAD/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ca4d1464b186e3d"}},{"code_sha256_prefix":"23fcd43811c3d64a","entry":"cleanup_translation","repo":"cfedermann/Appraise","repo_kind":"official","path":"appraise/create_beta16_xml.py","file_url":"https://github.com/cfedermann/Appraise/blob/HEAD/appraise/create_beta16_xml.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"23fcd43811c3d64a"}},{"code_sha256_prefix":"cddf640e4ba7fed4","entry":"compute_agreement_scores","repo":"cfedermann/Appraise","repo_kind":"official","path":"appraise/compute_agreement_scores.py","file_url":"https://github.com/cfedermann/Appraise/blob/HEAD/appraise/compute_agreement_scores.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cddf640e4ba7fed4"}},{"code_sha256_prefix":"c64a280e2461cff6","entry":"extract_system_ids_from_label","repo":"cfedermann/Appraise","repo_kind":"official","path":"appraise/compute_agreement_scores.py","file_url":"https://github.com/cfedermann/Appraise/blob/HEAD/appraise/compute_agreement_scores.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"c64a280e2461cff6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}