{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/overcoming-language-priors-with-self","title":"Overcoming Language Priors with Self-supervised Learning for Visual Question Answering","arxiv_id":"2012.11528","date":"2020-12-17","proceeding":null,"authors":["Xi Zhu","Zhendong Mao","Chunxiao Liu","Peng Zhang","Bin Wang","Yongdong Zhang"],"abstract":"Most Visual Question Answering (VQA) models suffer from the language prior problem, which is caused by inherent data biases. Specifically, VQA models tend to answer questions (e.g., what color is the banana?) based on the high-frequency answers (e.g., yellow) ignoring image contents. Existing approaches tackle this problem by creating delicate models or introducing additional visual annotations to reduce question dependency while strengthening image dependency. However, they are still subject to the language prior problem since the data biases have not been even alleviated. In this paper, we introduce a self-supervised learning framework to solve this problem. Concretely, we first automatically generate labeled data to balance the biased data, and propose a self-supervised auxiliary task to utilize the balanced data to assist the base VQA model to overcome language priors. Our method can compensate for the data biases by generating balanced data without introducing external annotations. Experimental results show that our method can significantly outperform the state-of-the-art, improving the overall accuracy from 49.50% to 57.59% on the most commonly used benchmark VQA-CP v2. In other words, we can increase the performance of annotation-based methods by 16% without using external annotations.","url_abs":"https://arxiv.org/abs/2012.11528v1","url_pdf":"https://arxiv.org/pdf/2012.11528v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"overcoming-language-priors-with-self","repo_url":"https://github.com/CrossmodalGroup/SSL-VQA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.11528","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.11528"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CrossmodalGroup/SSL-VQA","reach":null}],"summary":{"ran":5,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"2b0829c0325ec336","entry":"Att_3","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b0829c0325ec336"}},{"code_sha256_prefix":"c0aedef157f0caf4","entry":"FCNet","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0aedef157f0caf4"}},{"code_sha256_prefix":"b97bebdb4ff97926","entry":"QuestionEmbedding","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b97bebdb4ff97926"}},{"code_sha256_prefix":"78c2b60a61a6f6f3","entry":"SimpleClassifier","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78c2b60a61a6f6f3"}},{"code_sha256_prefix":"d5f763946d9d924d","entry":"WordEmbedding","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5f763946d9d924d"}},{"code_sha256_prefix":"c213b477dc6eef52","entry":"get_act","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c213b477dc6eef52"}},{"code_sha256_prefix":"26077f523336af12","entry":"get_norm","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26077f523336af12"}},{"code_sha256_prefix":"3c1f5fcc89b26ec8","entry":"Model","repo":"CrossmodalGroup/SSL-VQA","repo_kind":"official","path":"base_model.py","file_url":"https://github.com/CrossmodalGroup/SSL-VQA/blob/HEAD/base_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c1f5fcc89b26ec8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}