{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-ai-complete-question-answering-a-set","title":"Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks","arxiv_id":"1502.05698","date":"2015-02-19","proceeding":null,"authors":["Jason Weston","Antoine Bordes","Sumit Chopra","Alexander M. Rush","Bart van Merriënboer","Armand Joulin","Tomas Mikolov"],"abstract":"One long-term goal of machine learning research is to produce methods that\nare applicable to reasoning and natural language, in particular building an\nintelligent dialogue agent. To measure progress towards that goal, we argue for\nthe usefulness of a set of proxy tasks that evaluate reading comprehension via\nquestion answering. Our tasks measure understanding in several ways: whether a\nsystem is able to answer questions via chaining facts, simple induction,\ndeduction and many more. The tasks are designed to be prerequisites for any\nsystem that aims to be capable of conversing with a human. We believe many\nexisting learning systems can currently not solve them, and hence our aim is to\nclassify these tasks into skill sets, so that researchers can identify (and\nthen rectify) the failings of their systems. We also extend and improve the\nrecently introduced Memory Networks model, and show it is able to solve some,\nbut not all, of the tasks.","url_abs":"http://arxiv.org/abs/1502.05698v10","url_pdf":"http://arxiv.org/pdf/1502.05698v10.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/facebook/bAbI-tasks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/AlexKuhnle/ShapeWorld","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/HSabbar/Question-Answering","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/aykutaaykut/Memory-Networks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/benprano/Question-Answering-Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/booydar/babilong","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/cstghitpku/GateMemN2N","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/domluna/memn2n","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/facebookarchive/babi-tasks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/facebookresearch/ParlAI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/ishalyminov/memn2n","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/jojonki/MemoryNetworks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/kirubarajan/roft","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/kirubarajan/trick","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/nastasiaF/5DEEP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/nbansal90/bAbi_QA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/prashil2792/Question-Answering-System-Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/qapitan/babi-marcus","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/sayakbanerjee1999/Chat-Bot","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"towards-ai-complete-question-answering-a-set","repo_url":"https://github.com/uwnlp/qrn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1502.05698","atlas_url":"https://app.syntology.ai/?focus=1502.05698","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1502.05698"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/ParlAI","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/booydar/babilong","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kirubarajan/roft","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/domluna/memn2n","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/prashil2792/Question-Answering-System-Deep-Learning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ishalyminov/memn2n","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebook/bAbI-tasks","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/benprano/Question-Answering-Deep-Learning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HSabbar/Question-Answering","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sayakbanerjee1999/Chat-Bot","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qapitan/babi-marcus","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookarchive/babi-tasks","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nbansal90/bAbi_QA","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nastasiaF/5DEEP","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uwnlp/qrn","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aykutaaykut/Memory-Networks","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kirubarajan/trick","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AlexKuhnle/ShapeWorld","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cstghitpku/GateMemN2N","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jojonki/MemoryNetworks","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"unverified":2},"by_repo_kind":{"listed":{"samples":3,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"8abcd06d5ce60851","entry":"position_encoding","repo":"domluna/memn2n","repo_kind":"listed","path":"memn2n/memn2n.py","file_url":"https://github.com/domluna/memn2n/blob/HEAD/memn2n/memn2n.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8abcd06d5ce60851"}},{"code_sha256_prefix":"9d142ef2a5e220be","entry":"parse_stories","repo":"prashil2792/Question-Answering-System-Deep-Learning","repo_kind":"listed","path":"memorynetwork.py","file_url":"https://github.com/prashil2792/Question-Answering-System-Deep-Learning/blob/HEAD/memorynetwork.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9d142ef2a5e220be"}},{"code_sha256_prefix":"6d27ce4e6c139f45","entry":"tokenize","repo":"prashil2792/Question-Answering-System-Deep-Learning","repo_kind":"listed","path":"memorynetwork.py","file_url":"https://github.com/prashil2792/Question-Answering-System-Deep-Learning/blob/HEAD/memorynetwork.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6d27ce4e6c139f45"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}