{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-devil-is-in-the-details-tackling-unimodal","title":"The Devil Is in the Details: Tackling Unimodal Spurious Correlations for Generalizable Multimodal Reward Models","arxiv_id":"2503.03122","date":"2025-03-05","proceeding":null,"authors":["Zichao Li","Xueru Wen","Jie Lou","Yuqiu Ji","Yaojie Lu","Xianpei Han","Debing Zhang","Le Sun"],"abstract":"Multimodal Reward Models (MM-RMs) are crucial for aligning Large Language Models (LLMs) with human preferences, particularly as LLMs increasingly interact with multimodal data. However, we find that MM-RMs trained on existing datasets often struggle to generalize to out-of-distribution data due to their reliance on unimodal spurious correlations, primarily text-only shortcuts within the training distribution, which prevents them from leveraging true multimodal reward functions. To address this, we introduce a Shortcut-aware MM-RM learning algorithm that mitigates this issue by dynamically reweighting training samples, shifting the distribution toward better multimodal understanding, and reducing dependence on unimodal spurious correlations. Our experiments demonstrate significant improvements in generalization, downstream task performance, and scalability, establishing a more robust framework for multimodal reward modeling.","url_abs":"https://arxiv.org/abs/2503.03122v4","url_pdf":"https://arxiv.org/pdf/2503.03122v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-devil-is-in-the-details-tackling-unimodal","repo_url":"https://github.com/alignrm/Generalizable-MM-RM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.03122","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.03122"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alignrm/Generalizable-MM-RM","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"ec8988100a011957","entry":"make_batch_pairs","repo":"alignrm/generalizable-mm-rm","repo_kind":"official","path":"main/InternVL-2/train_rm.py","file_url":"https://github.com/alignrm/generalizable-mm-rm/blob/HEAD/main/InternVL-2/train_rm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec8988100a011957"}},{"code_sha256_prefix":"3ed28adb31fa7070","entry":"make_conv","repo":"alignrm/generalizable-mm-rm","repo_kind":"official","path":"main/InternVL-2/train_rm.py","file_url":"https://github.com/alignrm/generalizable-mm-rm/blob/HEAD/main/InternVL-2/train_rm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ed28adb31fa7070"}},{"code_sha256_prefix":"5ae774adfcd3f49e","entry":"split_model","repo":"alignrm/generalizable-mm-rm","repo_kind":"official","path":"main/InternVL-2/infer_rm.py","file_url":"https://github.com/alignrm/generalizable-mm-rm/blob/HEAD/main/InternVL-2/infer_rm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ae774adfcd3f49e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}