{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-module-networks","title":"Neural Module Networks","arxiv_id":"1511.02799","date":"2015-11-09","proceeding":"CVPR 2016 6","authors":["Jacob Andreas","Marcus Rohrbach","Trevor Darrell","Dan Klein"],"abstract":"Visual question answering is fundamentally compositional in nature---a\nquestion like \"where is the dog?\" shares substructure with questions like \"what\ncolor is the dog?\" and \"where is the cat?\" This paper seeks to simultaneously\nexploit the representational capacity of deep networks and the compositional\nlinguistic structure of questions. We describe a procedure for constructing and\nlearning *neural module networks*, which compose collections of jointly-trained\nneural \"modules\" into deep networks for question answering. Our approach\ndecomposes questions into their linguistic substructures, and uses these\nstructures to dynamically instantiate modular networks (with reusable\ncomponents for recognizing dogs, classifying colors, etc.). The resulting\ncompound networks are jointly trained. We evaluate our approach on two\nchallenging datasets for visual question answering, achieving state-of-the-art\nresults on both the VQA natural image dataset and a new dataset of complex\nquestions about abstract shapes.","url_abs":"http://arxiv.org/abs/1511.02799v4","url_pdf":"http://arxiv.org/pdf/1511.02799v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-module-networks","repo_url":"https://github.com/abril4416/Mod-Zero-VQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}},{"paper_slug":"neural-module-networks","repo_url":"https://github.com/jacobandreas/nmn2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[{"slug":"shapes-1","name":"SHAPES","full_name":"Swarm Heuristics based Adaptive and Penalized Estimation of Splines"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-question-answering-on-vqa-v1-test-dev","task":"Visual Question Answering (VQA)","dataset":"VQA v1 test-dev","model":"NMN+LSTM+FT","rank_in_archive_order":7,"of":7,"metrics":{"Accuracy":"58.6"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-vqa-v1-test-std","task":"Visual Question Answering (VQA)","dataset":"VQA v1 test-std","model":"NMN+LSTM+FT","rank_in_archive_order":6,"of":6,"metrics":{"Accuracy":"58.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.02799","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}