{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bendvlm-test-time-debiasing-of-vision","title":"BendVLM: Test-Time Debiasing of Vision-Language Embeddings","arxiv_id":"2411.04420","date":"2024-11-07","proceeding":null,"authors":["Walter Gerych","Haoran Zhang","Kimia Hamidieh","Eileen Pan","Maanas Sharma","Thomas Hartvigsen","Marzyeh Ghassemi"],"abstract":"Vision-language model (VLM) embeddings have been shown to encode biases present in their training data, such as societal biases that prescribe negative characteristics to members of various racial and gender identities. VLMs are being quickly adopted for a variety of tasks ranging from few-shot classification to text-guided image generation, making debiasing VLM embeddings crucial. Debiasing approaches that fine-tune the VLM often suffer from catastrophic forgetting. On the other hand, fine-tuning-free methods typically utilize a \"one-size-fits-all\" approach that assumes that correlation with the spurious attribute can be explained using a single linear direction across all possible inputs. In this work, we propose Bend-VLM, a nonlinear, fine-tuning-free approach for VLM embedding debiasing that tailors the debiasing operation to each unique input. This allows for a more flexible debiasing approach. Additionally, we do not require knowledge of the set of inputs a priori to inference time, making our method more appropriate for online, open-set tasks such as retrieval and text guided image generation.","url_abs":"https://arxiv.org/abs/2411.04420v1","url_pdf":"https://arxiv.org/pdf/2411.04420v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bendvlm-test-time-debiasing-of-vision","repo_url":"https://github.com/waltergerych/bend_vlm","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2411.04420","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.04420"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/waltergerych/bend_vlm","reach":{"status":"ok"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"3a8b1817d92c10aa","entry":"get_basic_template","repo":"waltergerych/bend_vlm","repo_kind":"official","path":"attribute_augment_component.py","file_url":"https://github.com/waltergerych/bend_vlm/blob/HEAD/attribute_augment_component.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3a8b1817d92c10aa"}},{"code_sha256_prefix":"050b743339edef0b","entry":"get_embeddings","repo":"waltergerych/bend_vlm","repo_kind":"official","path":"bend_utils.py","file_url":"https://github.com/waltergerych/bend_vlm/blob/HEAD/bend_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"050b743339edef0b"}},{"code_sha256_prefix":"59f8aa4adfe9fb8e","entry":"get_paraphrasings","repo":"waltergerych/bend_vlm","repo_kind":"official","path":"attribute_augment_component.py","file_url":"https://github.com/waltergerych/bend_vlm/blob/HEAD/attribute_augment_component.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59f8aa4adfe9fb8e"}},{"code_sha256_prefix":"2c6f3c826b749243","entry":"get_proj_matrix","repo":"waltergerych/bend_vlm","repo_kind":"official","path":"bend_utils.py","file_url":"https://github.com/waltergerych/bend_vlm/blob/HEAD/bend_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c6f3c826b749243"}},{"code_sha256_prefix":"d66aea8890d85881","entry":"replace_a_with_an","repo":"waltergerych/bend_vlm","repo_kind":"official","path":"attribute_augment_component.py","file_url":"https://github.com/waltergerych/bend_vlm/blob/HEAD/attribute_augment_component.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d66aea8890d85881"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}