{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/making-neural-qa-as-simple-as-possible-but","title":"Making Neural QA as Simple as Possible but not Simpler","arxiv_id":"1703.04816","date":"2017-03-14","proceeding":"CONLL 2017 8","authors":["Dirk Weissenborn","Georg Wiese","Laura Seiffe"],"abstract":"Recent development of large-scale question answering (QA) datasets triggered\na substantial amount of research into end-to-end neural architectures for QA.\nIncreasingly complex systems have been conceived without comparison to simpler\nneural baseline systems that would justify their complexity. In this work, we\npropose a simple heuristic that guides the development of neural baseline\nsystems for the extractive QA task. We find that there are two ingredients\nnecessary for building a high-performing neural QA system: first, the awareness\nof question words while processing the context and second, a composition\nfunction that goes beyond simple bag-of-words modeling, such as recurrent\nneural networks. Our results show that FastQA, a system that meets these two\nrequirements, can achieve very competitive performance compared with existing\nmodels. We argue that this surprising finding puts results of previous systems\nand the complexity of recent QA datasets into perspective.","url_abs":"http://arxiv.org/abs/1703.04816v3","url_pdf":"http://arxiv.org/pdf/1703.04816v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"making-neural-qa-as-simple-as-possible-but","repo_url":"https://github.com/newmast/QA-Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"making-neural-qa-as-simple-as-possible-but","repo_url":"https://github.com/uclmr/jack","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"making-neural-qa-as-simple-as-possible-but","repo_url":"https://github.com/uclnlp/jack","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-newsqa","task":"Question Answering","dataset":"NewsQA","model":"FastQAExt","rank_in_archive_order":15,"of":18,"metrics":{"EM":"43.7","F1":"56.1"},"uses_additional_data":true},{"leaderboard":"/sota/question-answering-on-squad11","task":"Question Answering","dataset":"SQuAD1.1","model":"FastQAExt","rank_in_archive_order":156,"of":213,"metrics":{"EM":"70.849","F1":"78.857"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad11","task":"Question Answering","dataset":"SQuAD1.1","model":"FastQA","rank_in_archive_order":166,"of":213,"metrics":{"EM":"68.436","F1":"77.070"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad11-dev","task":"Question Answering","dataset":"SQuAD1.1 dev","model":"FastQAExt (beam-size 5)","rank_in_archive_order":37,"of":55,"metrics":{"EM":"70.3","F1":"78.5"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1703.04816","atlas_url":"https://app.syntology.ai/?focus=1703.04816","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1703.04816"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uclnlp/jack","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/newmast/QA-Deep-Learning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uclmr/jack","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":1},"by_repo_kind":{"listed":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4961fc4669cf2576","entry":"jack_to_qasetting","repo":"uclmr/jack","repo_kind":"listed","path":"jack/core/data_structures.py","file_url":"https://github.com/uclmr/jack/blob/HEAD/jack/core/data_structures.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4961fc4669cf2576"}},{"code_sha256_prefix":"827e812c314c4426","entry":"fetch_parents","repo":"uclmr/jack","repo_kind":"listed","path":"bin/jack-train.py","file_url":"https://github.com/uclmr/jack/blob/HEAD/bin/jack-train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"827e812c314c4426"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}