{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-17918","title":"Benchmarking Direct Preference Optimization for Medical Large Vision-Language Models","arxiv_id":"2601.17918","date":"2026-01-25","proceeding":null,"authors":["Dain Kim","Jiwoo Lee","Jaehoon Yun","Yong Hoe Koo","Qingyu Chen","Hyunjae Kim","Jaewoo Kang"],"abstract":"Large Vision-Language Models (LVLMs) hold significant promise for medical applications, yet their deployment is often constrained by insufficient alignment and reliability. While Direct Preference Optimization (DPO) has emerged as a potent framework for refining model responses, its efficacy in high-stakes medical contexts remains underexplored, lacking the rigorous empirical groundwork necessary to guide future methodological advances. To bridge this gap, we present the first comprehensive examination of diverse DPO variants within the medical domain, evaluating nine distinct formulations across two medical LVLMs: LLaVA-Med and HuatuoGPT-Vision. Our results reveal several critical limitations: current DPO approaches often yield inconsistent gains over supervised fine-tuning, with their efficacy varying significantly across different tasks and backbones. Furthermore, they frequently fail to resolve fundamental visual misinterpretation errors. Building on these insights, we present a targeted preference construction strategy as a proof-of-concept that explicitly addresses visual misinterpretation errors frequently observed in existing DPO models. This design yields a 3.6% improvement over the strongest existing DPO baseline on visual question-answering tasks. To support future research, we release our complete framework, including all training data, model checkpoints, and our codebase at https://github.com/dmis-lab/med-vlm-dpo.","url_abs":"https://arxiv.org/abs/2601.17918","url_pdf":"https://arxiv.org/pdf/2601.17918","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2601.17918","atlas_url":"https://app.syntology.ai/?focus=2601.17918","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.17918"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/dmis-lab/med-vlm-dpo","reach":{"status":"ok"}}],"summary":{"ran":6,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"fcb74c8a86642376","entry":"compare_messages_gen","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/eval/eval_multimodal_chat_gpt_score.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/eval/eval_multimodal_chat_gpt_score.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fcb74c8a86642376"}},{"code_sha256_prefix":"c7eacf8297119313","entry":"conv_to_str","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/eval/eval_multimodal_chat_gpt_score.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/eval/eval_multimodal_chat_gpt_score.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7eacf8297119313"}},{"code_sha256_prefix":"908505ed68ff4871","entry":"expand2square","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/mm_utils.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/mm_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"908505ed68ff4871"}},{"code_sha256_prefix":"42a46570620cd9fa","entry":"get_chunk","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/eval/model_vqa.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/eval/model_vqa.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42a46570620cd9fa"}},{"code_sha256_prefix":"c3ee9d07c900dd55","entry":"load_image_from_base64","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/mm_utils.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/mm_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3ee9d07c900dd55"}},{"code_sha256_prefix":"f0f8c9c5a77c9b5b","entry":"process_images","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/mm_utils.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/mm_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0f8c9c5a77c9b5b"}},{"code_sha256_prefix":"076c252c52cbb161","entry":"split_list","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/eval/model_vqa.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/eval/model_vqa.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"076c252c52cbb161"}},{"code_sha256_prefix":"90719ea4ae0ed586","entry":"sum_list_list","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/eval/eval_multimodal_chat_gpt_score.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/eval/eval_multimodal_chat_gpt_score.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"90719ea4ae0ed586"}},{"code_sha256_prefix":"37899f22fb191b37","entry":"pretty_print_semaphore","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/utils.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37899f22fb191b37"}},{"code_sha256_prefix":"f9939a84b9a65279","entry":"violates_moderation","repo":"dmis-lab/med-vlm-dpo","repo_kind":"found_in_text","path":"inference/LLaVA-Med/llava/utils.py","file_url":"https://github.com/dmis-lab/med-vlm-dpo/blob/HEAD/inference/LLaVA-Med/llava/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9939a84b9a65279"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}