{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mediconfusion-can-you-trust-your-ai","title":"MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models","arxiv_id":"2409.15477","date":"2024-09-23","proceeding":null,"authors":["Mohammad Shahab Sepehri","Zalan Fabian","Maryam Soltanolkotabi","Mahdi Soltanolkotabi"],"abstract":"Multimodal Large Language Models (MLLMs) have tremendous potential to improve the accuracy, availability, and cost-effectiveness of healthcare by providing automated solutions or serving as aids to medical professionals. Despite promising first steps in developing medical MLLMs in the past few years, their capabilities and limitations are not well-understood. Recently, many benchmark datasets have been proposed that test the general medical knowledge of such models across a variety of medical areas. However, the systematic failure modes and vulnerabilities of such models are severely underexplored with most medical benchmarks failing to expose the shortcomings of existing models in this safety-critical domain. In this paper, we introduce MediConfusion, a challenging medical Visual Question Answering (VQA) benchmark dataset, that probes the failure modes of medical MLLMs from a vision perspective. We reveal that state-of-the-art models are easily confused by image pairs that are otherwise visually dissimilar and clearly distinct for medical experts. Strikingly, all available models (open-source or proprietary) achieve performance below random guessing on MediConfusion, raising serious concerns about the reliability of existing medical MLLMs for healthcare deployment. We also extract common patterns of model failure that may help the design of a new generation of more trustworthy and reliable MLLMs in healthcare.","url_abs":"https://arxiv.org/abs/2409.15477v2","url_pdf":"https://arxiv.org/pdf/2409.15477v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mediconfusion-can-you-trust-your-ai","repo_url":"https://github.com/mshahabsepehri/mediconfusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"mediconfusion-can-you-trust-your-ai","repo_url":"https://github.com/AIF4S/MediConfusion","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"medical-visual-question-answering","task_name":"Medical Visual Question Answering"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[{"slug":"mediconfusion","name":"MediConfusion","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.15477","atlas_url":"https://app.syntology.ai/?focus=2409.15477","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.15477"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/AIF4S/MediConfusion","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mshahabsepehri/mediconfusion","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":11,"ran_draft_wrong":1,"unverified":16},"by_repo_kind":{"official":{"samples":28,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":28,"samples":[{"code_sha256_prefix":"63d7c65dbf3e4810","entry":"combine_and_preprocess","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/radfm.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/radfm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"63d7c65dbf3e4810"}},{"code_sha256_prefix":"1e6bc9eb23773759","entry":"do_forward","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llava.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llava.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e6bc9eb23773759"}},{"code_sha256_prefix":"adf2ff1b0e7830cf","entry":"do_forward","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llava_med.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llava_med.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"adf2ff1b0e7830cf"}},{"code_sha256_prefix":"03bf30ed79071c64","entry":"do_generation","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llama.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llama.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03bf30ed79071c64"}},{"code_sha256_prefix":"a93ae912ac55bf0c","entry":"do_generation","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/molmo.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/molmo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a93ae912ac55bf0c"}},{"code_sha256_prefix":"f41cb1a19b154297","entry":"encode_image","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/gpt.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/gpt.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f41cb1a19b154297"}},{"code_sha256_prefix":"1db20718cb10a401","entry":"get_base64_encoded_image","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/claude.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/claude.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1db20718cb10a401"}},{"code_sha256_prefix":"096c2e54837bfa21","entry":"get_client","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/gpt.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"096c2e54837bfa21"}},{"code_sha256_prefix":"898e678f67126e4f","entry":"get_few_shot_sample","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/med_flamingo.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/med_flamingo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"898e678f67126e4f"}},{"code_sha256_prefix":"40fb3b39651a7e1e","entry":"get_response","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/gpt.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40fb3b39651a7e1e"}},{"code_sha256_prefix":"14187ee80a99236f","entry":"process_prompt","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/med_flamingo.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/med_flamingo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"14187ee80a99236f"}},{"code_sha256_prefix":"96c1cee40f32f817","entry":"split_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/deepseek.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/deepseek.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"96c1cee40f32f817"}},{"code_sha256_prefix":"ff4ebd2f87014a98","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/blip2.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/blip2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff4ebd2f87014a98"}},{"code_sha256_prefix":"0417998a0352dbaf","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/claude.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/claude.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0417998a0352dbaf"}},{"code_sha256_prefix":"235115d67de52abc","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/deepseek.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/deepseek.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"235115d67de52abc"}},{"code_sha256_prefix":"29f78d8995bbdd9c","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/instructblip.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/instructblip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"29f78d8995bbdd9c"}},{"code_sha256_prefix":"c0cf0aa63fb2b870","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llama.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0cf0aa63fb2b870"}},{"code_sha256_prefix":"cb4f65908c8c82e1","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llava.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llava.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb4f65908c8c82e1"}},{"code_sha256_prefix":"86480bca06e972e5","entry":"ask_question","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/molmo.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/molmo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86480bca06e972e5"}},{"code_sha256_prefix":"b072a5112146ab22","entry":"do_generation","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/blip2.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/blip2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b072a5112146ab22"}},{"code_sha256_prefix":"184ee47396ee9704","entry":"do_generation","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/instructblip.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/instructblip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"184ee47396ee9704"}},{"code_sha256_prefix":"5f189e3bd38f0f0c","entry":"get_tokenizer","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/radfm.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/radfm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f189e3bd38f0f0c"}},{"code_sha256_prefix":"56a36c6289b05241","entry":"load_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/blip2.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/blip2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56a36c6289b05241"}},{"code_sha256_prefix":"4396be49952addde","entry":"load_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/deepseek.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/deepseek.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4396be49952addde"}},{"code_sha256_prefix":"2fae4461a850afc1","entry":"load_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/instructblip.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/instructblip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2fae4461a850afc1"}},{"code_sha256_prefix":"d37a3a4453bba54d","entry":"load_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llama.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d37a3a4453bba54d"}},{"code_sha256_prefix":"12a456d721d6bf85","entry":"load_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/llava.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/llava.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12a456d721d6bf85"}},{"code_sha256_prefix":"a14ab87f7b097f4b","entry":"load_model","repo":"AIF4S/MediConfusion","repo_kind":"official","path":"Models/molmo.py","file_url":"https://github.com/AIF4S/MediConfusion/blob/HEAD/Models/molmo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a14ab87f7b097f4b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}