{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/minigpt-med-large-language-model-as-a-general","title":"MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis","arxiv_id":"2407.04106","date":"2024-07-04","proceeding":null,"authors":["Asma Alkhaldi","Raneem Alnajim","Layan Alabdullatef","Rawan Alyahya","Jun Chen","Deyao Zhu","Ahmed Alsinan","Mohamed Elhoseiny"],"abstract":"Recent advancements in artificial intelligence (AI) have precipitated significant breakthroughs in healthcare, particularly in refining diagnostic procedures. However, previous studies have often been constrained to limited functionalities. This study introduces MiniGPT-Med, a vision-language model derived from large-scale language models and tailored for medical applications. MiniGPT-Med demonstrates remarkable versatility across various imaging modalities, including X-rays, CT scans, and MRIs, enhancing its utility. The model is capable of performing tasks such as medical report generation, visual question answering (VQA), and disease identification within medical imagery. Its integrated processing of both image and textual clinical data markedly improves diagnostic accuracy. Our empirical assessments confirm MiniGPT-Med's superior performance in disease grounding, medical report generation, and VQA benchmarks, representing a significant step towards reducing the gap in assisting radiology practice. Furthermore, it achieves state-of-the-art performance on medical report generation, higher than the previous best model by 19\\% accuracy. MiniGPT-Med promises to become a general interface for radiology diagnoses, enhancing diagnostic efficiency across a wide range of medical imaging applications.","url_abs":"https://arxiv.org/abs/2407.04106v1","url_pdf":"https://arxiv.org/pdf/2407.04106v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"minigpt-med-large-language-model-as-a-general","repo_url":"https://github.com/vision-cair/minigpt-med","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"medical-report-generation","task_name":"Medical Report Generation"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.04106","atlas_url":"https://app.syntology.ai/?focus=2407.04106","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.04106"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vision-cair/minigpt-med","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran_fixture":2,"ran_violates":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"747d1d6a93d4f89d","entry":"list_of_str","repo":"vision-cair/minigpt-med","repo_kind":"official","path":"eval_scripts/model_evaluation.py","file_url":"https://github.com/vision-cair/minigpt-med/blob/HEAD/eval_scripts/model_evaluation.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"747d1d6a93d4f89d"}},{"code_sha256_prefix":"f8e4c1c5aac977d6","entry":"computeIoU","repo":"vision-cair/minigpt-med","repo_kind":"official","path":"demo_gradio4-29.py","file_url":"https://github.com/vision-cair/minigpt-med/blob/HEAD/demo_gradio4-29.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8e4c1c5aac977d6"}},{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"vision-cair/minigpt-med","repo_kind":"official","path":"minigpt4/models/base_model.py","file_url":"https://github.com/vision-cair/minigpt-med/blob/HEAD/minigpt4/models/base_model.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"5c3e6655c3b3e958","entry":"extract_substrings","repo":"vision-cair/minigpt-med","repo_kind":"official","path":"demo_gradio4-29.py","file_url":"https://github.com/vision-cair/minigpt-med/blob/HEAD/demo_gradio4-29.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5c3e6655c3b3e958"}},{"code_sha256_prefix":"37438be3369e6493","entry":"is_overlapping","repo":"vision-cair/minigpt-med","repo_kind":"official","path":"demo_gradio4-29.py","file_url":"https://github.com/vision-cair/minigpt-med/blob/HEAD/demo_gradio4-29.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37438be3369e6493"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}