{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/debiasing-multimodal-large-language-models","title":"Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization","arxiv_id":"2503.17928","date":"2025-03-23","proceeding":"CVPR 2025 1","authors":["Zefeng Zhang","Hengzhu Tang","Jiawei Sheng","Zhenyu Zhang","Yiming Ren","Zhenyang Li","Dawei Yin","Duohe Ma","Tingwen Liu"],"abstract":"Multimodal Large Language Models excel in various tasks, yet often struggle with modality bias, where the model tends to rely heavily on a single modality and overlook critical information in other modalities, which leads to incorrect focus and generating irrelevant responses. In this paper, we propose using the paradigm of preference optimization to solve the modality bias problem, including RLAIFVBias, a debiased preference optimization dataset, and a Noise Aware Preference Optimization algorithm. Specifically, we first construct the dataset by introducing perturbations to reduce the informational content of certain modalities, compelling the model to rely on a specific modality when generating negative responses. To address the inevitable noise in automatically constructed data, we combine the noise robust Mean Absolute Error with the Binary Cross Entropy in Direct Preference Optimization by a negative Box Cox transformation, and dynamically adjust the algorithm noise robustness based on the evaluated noise levels in the data. Extensive experiments validate our approach, demonstrating not only its effectiveness in mitigating modality bias but also its significant role in minimizing hallucinations.","url_abs":"https://arxiv.org/abs/2503.17928v1","url_pdf":"https://arxiv.org/pdf/2503.17928v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"debiasing-multimodal-large-language-models","repo_url":"https://github.com/zhangzef/NaPO","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2503.17928","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.17928"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zhangzef/NaPO","reach":null}],"summary":{"ran_draft_wrong":3,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"b7bdef9e35a830ec","entry":"_tokenize_fn","repo":"zhangzef/NaPO","repo_kind":"official","path":"muffin/train/train_utils.py","file_url":"https://github.com/zhangzef/NaPO/blob/HEAD/muffin/train/train_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7bdef9e35a830ec"}},{"code_sha256_prefix":"9813e877cbde1088","entry":"encode_multimodal_preference_sample","repo":"zhangzef/NaPO","repo_kind":"official","path":"muffin/train/train_utils.py","file_url":"https://github.com/zhangzef/NaPO/blob/HEAD/muffin/train/train_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9813e877cbde1088"}},{"code_sha256_prefix":"3521cb6c565d2946","entry":"expand_image_token","repo":"zhangzef/NaPO","repo_kind":"official","path":"muffin/train/train_utils.py","file_url":"https://github.com/zhangzef/NaPO/blob/HEAD/muffin/train/train_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3521cb6c565d2946"}},{"code_sha256_prefix":"2c418561e94d2c3c","entry":"_add_speaker_and_signal","repo":"zhangzef/NaPO","repo_kind":"official","path":"muffin/train/train_utils.py","file_url":"https://github.com/zhangzef/NaPO/blob/HEAD/muffin/train/train_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c418561e94d2c3c"}},{"code_sha256_prefix":"b63a88fb0459f878","entry":"_mask_targets","repo":"zhangzef/NaPO","repo_kind":"official","path":"muffin/train/train_utils.py","file_url":"https://github.com/zhangzef/NaPO/blob/HEAD/muffin/train/train_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b63a88fb0459f878"}},{"code_sha256_prefix":"8b6b2600164187a9","entry":"preprocess","repo":"zhangzef/NaPO","repo_kind":"official","path":"muffin/train/train_utils.py","file_url":"https://github.com/zhangzef/NaPO/blob/HEAD/muffin/train/train_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8b6b2600164187a9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}