{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-simple-loss-function-for-improving-the","title":"A Simple Loss Function for Improving the Convergence and Accuracy of Visual Question Answering Models","arxiv_id":"1708.00584","date":"2017-08-02","proceeding":null,"authors":["Ilija Ilievski","Jiashi Feng"],"abstract":"Visual question answering as recently proposed multimodal learning task has\nenjoyed wide attention from the deep learning community. Lately, the focus was\non developing new representation fusion methods and attention mechanisms to\nachieve superior performance. On the other hand, very little focus has been put\non the models' loss function, arguably one of the most important aspects of\ntraining deep learning models. The prevailing practice is to use cross entropy\nloss function that penalizes the probability given to all the answers in the\nvocabulary except the single most common answer for the particular question.\nHowever, the VQA evaluation function compares the predicted answer with all the\nground-truth answers for the given question and if there is a matching, a\npartial point is given. This causes a discrepancy between the model's cross\nentropy loss and the model's accuracy as calculated by the VQA evaluation\nfunction. In this work, we propose a novel loss, termed as soft cross entropy,\nthat considers all ground-truth answers and thus reduces the loss-accuracy\ndiscrepancy. The proposed loss leads to an improved training convergence of VQA\nmodels and an increase in accuracy as much as 1.6%.","url_abs":"http://arxiv.org/abs/1708.00584v1","url_pdf":"http://arxiv.org/pdf/1708.00584v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-simple-loss-function-for-improving-the","repo_url":"https://github.com/ilija139/vqa-soft","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"torch","reach":null},{"paper_slug":"a-simple-loss-function-for-improving-the","repo_url":"https://github.com/abhijit-buet/VizWiz-Visua-Question-Answering-2021","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"a-simple-loss-function-for-improving-the","repo_url":"https://github.com/abhijit-buet/VizWiz-Visual-Question-Answering-2021","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}