{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/image-question-answering-using-convolutional","title":"Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction","arxiv_id":"1511.05756","date":"2015-11-18","proceeding":"CVPR 2016 6","authors":["Hyeonwoo Noh","Paul Hongsuck Seo","Bohyung Han"],"abstract":"We tackle image question answering (ImageQA) problem by learning a\nconvolutional neural network (CNN) with a dynamic parameter layer whose weights\nare determined adaptively based on questions. For the adaptive parameter\nprediction, we employ a separate parameter prediction network, which consists\nof gated recurrent unit (GRU) taking a question as its input and a\nfully-connected layer generating a set of candidate weights as its output.\nHowever, it is challenging to construct a parameter prediction network for a\nlarge number of parameters in the fully-connected dynamic parameter layer of\nthe CNN. We reduce the complexity of this problem by incorporating a hashing\ntechnique, where the candidate weights given by the parameter prediction\nnetwork are selected using a predefined hash function to determine individual\nweights in the dynamic parameter layer. The proposed network---joint network\nwith the CNN for ImageQA and the parameter prediction network---is trained\nend-to-end through back-propagation, where its weights are initialized using a\npre-trained CNN and GRU. The proposed algorithm illustrates the\nstate-of-the-art performance on all available public ImageQA benchmarks.","url_abs":"http://arxiv.org/abs/1511.05756v1","url_pdf":"http://arxiv.org/pdf/1511.05756v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"image-question-answering-using-convolutional","repo_url":"https://github.com/HyeonwooNoh/DPPnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"torch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"multi-modal","task_name":"Image Retrieval with Multi-Modal Query"},{"task_slug":"parameter-prediction","task_name":"Parameter Prediction"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[{"method_slug":"gru","method_name":"GRU"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-retrieval-with-multi-modal-query-on","task":"Image Retrieval with Multi-Modal Query","dataset":"Fashion200k","model":"Param Hashing","rank_in_archive_order":7,"of":8,"metrics":{"Recall@1":"12.2","Recall@10":"40","Recall@50":"61.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.05756","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}