{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/see-it-from-my-perspective-diagnosing-the","title":"See It from My Perspective: Diagnosing the Western Cultural Bias of Large Vision-Language Models in Image Understanding","arxiv_id":"2406.11665","date":"2024-06-17","proceeding":null,"authors":["Amith Ananthram","Elias Stengel-Eskin","Carl Vondrick","Mohit Bansal","Kathleen McKeown"],"abstract":"Vision-language models (VLMs) can respond to queries about images in many languages. However, beyond language, culture affects how we see things. For example, individuals from Western cultures focus more on the central figure in an image while individuals from Eastern cultures attend more to scene context. In this work, we present a novel investigation that demonstrates and localizes VLMs' Western bias in image understanding. We evaluate large VLMs across subjective and objective visual tasks with culturally diverse images and annotations. We find that VLMs perform better on the Western subset than the Eastern subset of each task. Controlled experimentation tracing the source of this bias highlights the importance of a diverse language mix in text-only pre-training for building equitable VLMs, even when inference is performed in English. Moreover, while prompting in the language of a target culture can lead to reductions in bias, it is not a substitute for building AI more representative of the world's languages.","url_abs":"https://arxiv.org/abs/2406.11665v1","url_pdf":"https://arxiv.org/pdf/2406.11665v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"see-it-from-my-perspective-diagnosing-the","repo_url":"https://github.com/amith-ananthram/see-it-from-my-perspective","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.11665","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.11665"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amith-ananthram/see-it-from-my-perspective","reach":{"status":"ok"}}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"db19112cf3afad4c","entry":"clean_generation","repo":"amith-ananthram/see-it-from-my-perspective","repo_kind":"official","path":"utils.py","file_url":"https://github.com/amith-ananthram/see-it-from-my-perspective/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"db19112cf3afad4c"}},{"code_sha256_prefix":"ff3df9199f8ff9b5","entry":"get_output_key","repo":"amith-ananthram/see-it-from-my-perspective","repo_kind":"official","path":"3_judge_dollarstreet_with_prometheus.py","file_url":"https://github.com/amith-ananthram/see-it-from-my-perspective/blob/HEAD/3_judge_dollarstreet_with_prometheus.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff3df9199f8ff9b5"}},{"code_sha256_prefix":"2d9478181954b86e","entry":"parse_selection","repo":"amith-ananthram/see-it-from-my-perspective","repo_kind":"official","path":"2_extract_mcvqa_artelingo_selections.py","file_url":"https://github.com/amith-ananthram/see-it-from-my-perspective/blob/HEAD/2_extract_mcvqa_artelingo_selections.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2d9478181954b86e"}},{"code_sha256_prefix":"d065eeaf46e1419c","entry":"prepare_inputs","repo":"amith-ananthram/see-it-from-my-perspective","repo_kind":"official","path":"3_judge_dollarstreet_with_prometheus.py","file_url":"https://github.com/amith-ananthram/see-it-from-my-perspective/blob/HEAD/3_judge_dollarstreet_with_prometheus.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d065eeaf46e1419c"}},{"code_sha256_prefix":"f39913d7cb0b3a04","entry":"prompt_collator","repo":"amith-ananthram/see-it-from-my-perspective","repo_kind":"official","path":"1_run_vlm_cultural_bias_inference.py","file_url":"https://github.com/amith-ananthram/see-it-from-my-perspective/blob/HEAD/1_run_vlm_cultural_bias_inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f39913d7cb0b3a04"}},{"code_sha256_prefix":"d5d54c2f4ad20f9e","entry":"load_to_process","repo":"amith-ananthram/see-it-from-my-perspective","repo_kind":"official","path":"3_judge_dollarstreet_with_prometheus.py","file_url":"https://github.com/amith-ananthram/see-it-from-my-perspective/blob/HEAD/3_judge_dollarstreet_with_prometheus.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5d54c2f4ad20f9e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}