{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mia-dpo-multi-image-augmented-direct","title":"MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models","arxiv_id":"2410.17637","date":"2024-10-23","proceeding":null,"authors":["Ziyu Liu","Yuhang Zang","Xiaoyi Dong","Pan Zhang","Yuhang Cao","Haodong Duan","Conghui He","Yuanjun Xiong","Dahua Lin","Jiaqi Wang"],"abstract":"Visual preference alignment involves training Large Vision-Language Models (LVLMs) to predict human preferences between visual inputs. This is typically achieved by using labeled datasets of chosen/rejected pairs and employing optimization algorithms like direct preference optimization (DPO). Existing visual alignment methods, primarily designed for single-image scenarios, struggle to effectively handle the complexity of multi-image tasks due to the scarcity of diverse training data and the high cost of annotating chosen/rejected pairs. We present Multi-Image Augmented Direct Preference Optimization (MIA-DPO), a visual preference alignment approach that effectively handles multi-image inputs. MIA-DPO mitigates the scarcity of diverse multi-image training data by extending single-image data with unrelated images arranged in grid collages or pic-in-pic formats, significantly reducing the costs associated with multi-image data annotations. Our observation reveals that attention values of LVLMs vary considerably across different images. We use attention values to identify and filter out rejected responses the model may have mistakenly focused on. Our attention-aware selection for constructing the chosen/rejected pairs without relying on (i) human annotation, (ii) extra data, and (iii) external models or APIs. MIA-DPO is compatible with various architectures and outperforms existing methods on five multi-image benchmarks, achieving an average performance boost of 3.0% on LLaVA-v1.5 and 4.3% on the recent InternLM-XC2.5. Moreover, MIA-DPO has a minimal effect on the model's ability to understand single images.","url_abs":"https://arxiv.org/abs/2410.17637v1","url_pdf":"https://arxiv.org/pdf/2410.17637v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mia-dpo-multi-image-augmented-direct","repo_url":"https://github.com/liuziyu77/mia-dpo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.17637","atlas_url":"https://app.syntology.ai/?focus=2410.17637","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.17637"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liuziyu77/mia-dpo","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":3,"ran":1,"ran_violates":1,"unverified":6},"by_repo_kind":{"official":{"samples":9,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"f1a2ea781d07a5ee","entry":"get_id_from_path","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f1a2ea781d07a5ee"}},{"code_sha256_prefix":"c32b636f13ed9ee5","entry":"convert_to_scalar","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/llava_hound_dpo/trl/core.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/llava_hound_dpo/trl/core.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c32b636f13ed9ee5"}},{"code_sha256_prefix":"42a46570620cd9fa","entry":"get_chunk","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/chatuniv/ChatUniVi/eval/model_coco_vqa.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/chatuniv/ChatUniVi/eval/model_coco_vqa.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42a46570620cd9fa"}},{"code_sha256_prefix":"4354061aab23f5b9","entry":"remove_special_tokens","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4354061aab23f5b9"}},{"code_sha256_prefix":"076c252c52cbb161","entry":"split_list","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/chatuniv/ChatUniVi/eval/model_coco_vqa.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/chatuniv/ChatUniVi/eval/model_coco_vqa.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"076c252c52cbb161"}},{"code_sha256_prefix":"a51ae45a1f6c0b5b","entry":"flatten_dict","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/llava_hound_dpo/trl/core.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/llava_hound_dpo/trl/core.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a51ae45a1f6c0b5b"}},{"code_sha256_prefix":"fccf1d01505895d0","entry":"get_seq_frames","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/chatuniv/chatuniv_utils.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/chatuniv/chatuniv_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fccf1d01505895d0"}},{"code_sha256_prefix":"663e3b5a6740fa0d","entry":"remove_special_tokens","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/chatuniv/chatuniv_utils.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/chatuniv/chatuniv_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"663e3b5a6740fa0d"}},{"code_sha256_prefix":"160197f694bfebd8","entry":"remove_special_tokens","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/llama_vid/llamavid_utils.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/llama_vid/llamavid_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"160197f694bfebd8"}},{"code_sha256_prefix":"c3f1f1085b84693b","entry":"resize_image","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/chatuniv/chatuniv_utils.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/chatuniv/chatuniv_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3f1f1085b84693b"}},{"code_sha256_prefix":"085e70e0bd6e492d","entry":"stack_dicts","repo":"liuziyu77/mia-dpo","repo_kind":"official","path":"LLaVA-Hound-DPO/llava_hound_dpo/trl/core.py","file_url":"https://github.com/liuziyu77/mia-dpo/blob/HEAD/LLaVA-Hound-DPO/llava_hound_dpo/trl/core.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"085e70e0bd6e492d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}