{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mam-modular-multi-agent-framework-for-multi","title":"MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration","arxiv_id":"2506.19835","date":"2025-06-24","proceeding":null,"authors":["Yucheng Zhou","Lingran Song","Jianbing Shen"],"abstract":"Recent advancements in medical Large Language Models (LLMs) have showcased their powerful reasoning and diagnostic capabilities. Despite their success, current unified multimodal medical LLMs face limitations in knowledge update costs, comprehensiveness, and flexibility. To address these challenges, we introduce the Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis (MAM). Inspired by our empirical findings highlighting the benefits of role assignment and diagnostic discernment in LLMs, MAM decomposes the medical diagnostic process into specialized roles: a General Practitioner, Specialist Team, Radiologist, Medical Assistant, and Director, each embodied by an LLM-based agent. This modular and collaborative framework enables efficient knowledge updates and leverages existing medical LLMs and knowledge bases. Extensive experimental evaluations conducted on a wide range of publicly accessible multimodal medical datasets, incorporating text, image, audio, and video modalities, demonstrate that MAM consistently surpasses the performance of modality-specific LLMs. Notably, MAM achieves significant performance improvements ranging from 18% to 365% compared to baseline models. Our code is released at https://github.com/yczhou001/MAM.","url_abs":"https://arxiv.org/abs/2506.19835v1","url_pdf":"https://arxiv.org/pdf/2506.19835v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mam-modular-multi-agent-framework-for-multi","repo_url":"https://github.com/yczhou001/mam","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"medical-diagnosis","task_name":"Medical Diagnosis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.19835","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.19835"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yczhou001/mam","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"29e49122fb6492d3","entry":"final_diagnosis","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/diagnosis.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/diagnosis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29e49122fb6492d3"}},{"code_sha256_prefix":"fd78af923d0ea260","entry":"generate_role","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/role_generation.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/role_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd78af923d0ea260"}},{"code_sha256_prefix":"c7a6850f9c6bbf12","entry":"get_diagnosis_prompt","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/diagnosis.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/diagnosis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7a6850f9c6bbf12"}},{"code_sha256_prefix":"9ed7060ec4c112c9","entry":"get_discuss_prompt","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/meeting.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/meeting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ed7060ec4c112c9"}},{"code_sha256_prefix":"994211ad4c8cdf07","entry":"get_review_prompt","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/review.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/review.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"994211ad4c8cdf07"}},{"code_sha256_prefix":"6cdb9df2960b6e19","entry":"get_summarize_prompt","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/meeting.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/meeting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cdb9df2960b6e19"}},{"code_sha256_prefix":"903964d493cf4c43","entry":"get_vote_prompt","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/meeting.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/meeting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"903964d493cf4c43"}},{"code_sha256_prefix":"65fadffaf83373ed","entry":"load_history","repo":"yczhou001/mam","repo_kind":"official","path":"inference.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65fadffaf83373ed"}},{"code_sha256_prefix":"2b3be5367be96643","entry":"memory","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/memory.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/memory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b3be5367be96643"}},{"code_sha256_prefix":"6a34190e1793681b","entry":"modality_selection","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/modality_selection.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/modality_selection.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a34190e1793681b"}},{"code_sha256_prefix":"314f530e9a96dcc2","entry":"parse_type_result","repo":"yczhou001/mam","repo_kind":"official","path":"inference.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"314f530e9a96dcc2"}},{"code_sha256_prefix":"656a03356c53125b","entry":"review_all","repo":"yczhou001/mam","repo_kind":"official","path":"pipeline/review.py","file_url":"https://github.com/yczhou001/mam/blob/HEAD/pipeline/review.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"656a03356c53125b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}