{"url":"/task/medical-report-generation","name":"Medical Report Generation","slug":"medical-report-generation","description_markdown":"Medical report generation (MRG) is a task which focus on training AI to automatically generate professional report according the input image data. This can help clinicians make faster and more accurate decision since the task itself is both time consuming and error prone even for experienced doctors.\r\n\r\nAggfgg\r\n\r\nDeep neural network and transformer based architecture are currently the most popular methods for this certain task, however, when we try to transfer out pre-trained model into this certain domain, their performance always degrade. \r\n\r\n\r\n\r\nThe following are some of the reasons why RSG is hard for pre-trained models:\r\n\r\n* Language datasets in a particular domain can sometimes be quite different from the large number of datasets available on the Internet\r\n* During the fine-tuning phase, datasets in the medical field are often unevenly distributed\r\n\r\n\r\n\r\nMore recently, multi-modal learning and contrastive learning have shown some inspiring results in this field, but it's still challenging and requires further attention.\r\n\r\n\r\n\r\nHere are some additional readings to go deeper on the task:\r\n\r\n* On the Automatic Generation of Medical Imaging Reports\r\n\r\n  [ https://doi.org/10.48550/arXiv.1711.08195](https://doi.org/10.48550/arXiv.1711.08195)\r\n\r\n* A scoping review of transfer learning research on medical image analysis using ImageNet \r\n\r\n  [ https://arxiv.org/abs/2004.13175](https://arxiv.org/abs/2004.13175)\r\n\r\n* A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis\r\n\r\n [ https://arxiv.org/abs/2004.12150]( https://arxiv.org/abs/2004.12150)\r\n\r\n\r\n\r\n\r\n\r\n(Image credit : Transformers in Medical Imaging: A Survey)","categories":[{"name":"Medical","url":"/area/medical"},{"name":"Methodology","url":"/area/methodology"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":110,"papers_with_code":46,"benchmarks":3,"benchmark_tables_in_archive":3,"benchmark_tables_shown":3,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":9,"subtasks":0,"parent_tasks":0},"benchmarks":[{"leaderboard":"/sota/medical-report-generation-on-mimic-cxr","slug":"medical-report-generation-on-mimic-cxr","dataset":"MIMIC-CXR","dataset_url":"/dataset/mimic-cxr","rows_in_archive":2,"metrics":["BLEU-1","BLEU-2","BLEU-3","BLEU-4","CIDEr","Example-F1-14","Example-Precision-14","Example-Recall-14","METEOR","Micro-F1-5","Micro-Precision-5","Micro-Recall-5","ROUGE-L","F1 RadGraph"],"first_row_in_archive_order":{"model":"RGRG","paper_title":"Interactive and Explainable Region-guided Radiology Report Generation","paper_url":"/paper/interactive-and-explainable-region-guided","paper_date":"2023-04-17","arxiv_id":"2304.08295","code_links":[{"title":"ttanida/rgrg","url":"https://github.com/ttanida/rgrg"}],"syntology":{"n":10,"n_ran":0,"n_unverified":10,"n_pointer_only":0}}},{"leaderboard":"/sota/medical-report-generation-on-histgen-wsi","slug":"medical-report-generation-on-histgen-wsi","dataset":"HistGen WSI-Report Dataset","dataset_url":"/dataset/histgen-wsi-report-dataset","rows_in_archive":1,"metrics":["BLEU-4"],"first_row_in_archive_order":{"model":"HistGen","paper_title":"HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction","paper_url":"/paper/histgen-histopathology-report-generation-via","paper_date":"2024-03-08","arxiv_id":"2403.05396","code_links":[{"title":"dddavid4real/HistGen","url":"https://github.com/dddavid4real/HistGen"}],"syntology":{"n":6,"n_ran":6,"n_unverified":0,"n_pointer_only":0}}},{"leaderboard":"/sota/medical-report-generation-on-iu-x-ray","slug":"medical-report-generation-on-iu-x-ray","dataset":"IU X-Ray","dataset_url":"/dataset/iu-x-ray","rows_in_archive":1,"metrics":["BLEU-4","BLEU-1","BLEU-2","BLEU-3","CIDEr","METEOR","ROUGE"],"first_row_in_archive_order":{"model":"X-RGen","paper_title":"Act Like a Radiologist: Radiology Report Generation across Anatomical Regions","paper_url":"/paper/s4m-generating-radiology-reports-by-a-single","paper_date":"2023-05-26","arxiv_id":"2305.16685","code_links":[{"title":"ytongxie/x-rgen","url":"https://github.com/ytongxie/x-rgen"},{"title":"ytongxie/s4m","url":"https://github.com/ytongxie/s4m"}],"syntology":null}}],"datasets":[{"url":"/dataset/mimic-cxr","name":"MIMIC-CXR","full_name":"MIMIC-CXR","num_papers_in_archive":240},{"url":"/dataset/pathvqa","name":"PathVQA","full_name":"","num_papers_in_archive":89},{"url":"/dataset/iu-x-ray","name":"IU X-Ray","full_name":"","num_papers_in_archive":19},{"url":"/dataset/histgen-wsi-report-dataset","name":"HistGen WSI-Report Dataset","full_name":"","num_papers_in_archive":1},{"url":"/dataset/llava-rad-mimic-cxr-annotations","name":"LLaVA-Rad MIMIC-CXR Annotations","full_name":"","num_papers_in_archive":1},{"url":"/dataset/medtrinity-25m","name":"MedTrinity-25M","full_name":"","num_papers_in_archive":1},{"url":"/dataset/rate-ner","name":"RaTE-NER","full_name":"","num_papers_in_archive":1},{"url":"/dataset/smr-iu-x-ray","name":"SMR IU X-Ray","full_name":"Simplified Medical Reports","num_papers_in_archive":1},{"url":"/dataset/an-open-chest-x-ray-dataset-with-benchmarks","name":"CASIA-CXR","full_name":"","num_papers_in_archive":0}],"subtasks":[],"parent_tasks":[],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":46,"tagged_in_all":110,"items":[{"url":"/paper/on-the-automatic-generation-of-medical","title":"On the Automatic Generation of Medical Imaging Reports","date":"2017-11-22","arxiv_id":"1711.08195","repositories_listed":4,"syntology":null},{"url":"/paper/causalvlr-a-toolbox-and-benchmark-for-visual","title":"CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal Reasoning","date":"2023-06-30","arxiv_id":"2306.17462","repositories_listed":2,"syntology":null},{"url":"/paper/s4m-generating-radiology-reports-by-a-single","title":"Act Like a Radiologist: Radiology Report Generation across Anatomical Regions","date":"2023-05-26","arxiv_id":"2305.16685","repositories_listed":2,"syntology":null},{"url":"/paper/visual-linguistic-causal-intervention-for","title":"Cross-Modal Causal Intervention for Medical Report Generation","date":"2023-03-16","arxiv_id":"2303.09117","repositories_listed":2,"syntology":null},{"url":"/paper/auxiliary-signal-guided-knowledge-encoder","title":"Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation","date":"2020-06-06","arxiv_id":"2006.03744","repositories_listed":2,"syntology":null},{"url":"/paper/retrieval-augmented-generation-and-1","title":"Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook","date":"2025-03-23","arxiv_id":"2503.18016","repositories_listed":1,"syntology":null},{"url":"/paper/umit-unifying-medical-imaging-tasks-via","title":"UMIT: Unifying Medical Imaging Tasks via Vision-Language Models","date":"2025-03-20","arxiv_id":"2503.15892","repositories_listed":1,"syntology":null},{"url":"/paper/gema-score-granular-explainable-multi-agent","title":"GEMA-Score: Granular Explainable Multi-Agent Score for Radiology Report Evaluation","date":"2025-03-07","arxiv_id":"2503.05347","repositories_listed":1,"syntology":null},{"url":"/paper/activating-associative-disease-aware-vision","title":"Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation","date":"2025-01-07","arxiv_id":"2501.03458","repositories_listed":1,"syntology":null},{"url":"/paper/automated-medical-report-generation-for-ecg","title":"Automated Medical Report Generation for ECG Data: Bridging Medical Text and Signal Processing with Deep Learning","date":"2024-12-05","arxiv_id":"2412.04067","repositories_listed":1,"syntology":null},{"url":"/paper/a-survey-of-medical-vision-and-language","title":"A Survey of Medical Vision-and-Language Applications and Their Techniques","date":"2024-11-19","arxiv_id":"2411.12195","repositories_listed":1,"syntology":null},{"url":"/paper/cxpmrg-bench-pre-training-and-benchmarking","title":"CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset","date":"2024-10-01","arxiv_id":"2410.00379","repositories_listed":1,"syntology":{"n":4,"n_ran":4,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/r2gencsr-retrieving-context-samples-for-large","title":"R2GenCSR: Retrieving Context Samples for Large Language Model based X-ray Medical Report Generation","date":"2024-08-19","arxiv_id":"2408.09743","repositories_listed":1,"syntology":null},{"url":"/paper/ecg-chat-a-large-ecg-language-model-for","title":"ECG-Chat: A Large ECG-Language Model for Cardiac Disease Diagnosis","date":"2024-08-16","arxiv_id":"2408.08849","repositories_listed":1,"syntology":{"n":14,"n_ran":5,"n_unverified":9,"n_pointer_only":14}},{"url":"/paper/minigpt-med-large-language-model-as-a-general","title":"MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis","date":"2024-07-04","arxiv_id":"2407.04106","repositories_listed":1,"syntology":{"n":5,"n_ran":5,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/towards-a-holistic-framework-for-multimodal","title":"Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation","date":"2024-07-02","arxiv_id":"2407.02235","repositories_listed":1,"syntology":null},{"url":"/paper/structural-entities-extraction-and-patient","title":"Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation","date":"2024-05-23","arxiv_id":"2405.14905","repositories_listed":1,"syntology":null},{"url":"/paper/factual-serialization-enhancement-a-key","title":"Factual Serialization Enhancement: A Key Innovation for Chest X-ray Report Generation","date":"2024-05-15","arxiv_id":"2405.09586","repositories_listed":1,"syntology":null},{"url":"/paper/meddr-diagnosis-guided-bootstrapping-for","title":"GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist Collaboration","date":"2024-04-23","arxiv_id":"2404.15127","repositories_listed":1,"syntology":{"n":8,"n_ran":7,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/histgen-histopathology-report-generation-via","title":"HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction","date":"2024-03-08","arxiv_id":"2403.05396","repositories_listed":1,"syntology":{"n":6,"n_ran":6,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/vision-language-models-for-medical-report","title":"Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review","date":"2024-03-04","arxiv_id":"2403.02469","repositories_listed":1,"syntology":null},{"url":"/paper/icon-improving-inter-report-consistency-of","title":"ICON: Improving Inter-Report Consistency in Radiology Report Generation via Lesion-aware Mixup Augmentation","date":"2024-02-20","arxiv_id":"2402.12844","repositories_listed":1,"syntology":null},{"url":"/paper/pefomed-parameter-efficient-fine-tuning-on","title":"PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging","date":"2024-01-05","arxiv_id":"2401.02797","repositories_listed":1,"syntology":null},{"url":"/paper/improving-medical-report-generation-with","title":"Improving Medical Report Generation with Adapter Tuning and Knowledge Enhancement in Vision-Language Foundation Models","date":"2023-12-07","arxiv_id":"2312.03970","repositories_listed":1,"syntology":null},{"url":"/paper/complex-organ-mask-guided-radiology-report","title":"Complex Organ Mask Guided Radiology Report Generation","date":"2023-11-04","arxiv_id":"2311.02329","repositories_listed":1,"syntology":null},{"url":"/paper/recap-towards-precise-radiology-report","title":"RECAP: Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoning","date":"2023-10-21","arxiv_id":"2310.13864","repositories_listed":1,"syntology":null},{"url":"/paper/promptmrg-diagnosis-driven-prompts-for","title":"PromptMRG: Diagnosis-Driven Prompts for Medical Report Generation","date":"2023-08-24","arxiv_id":"2308.12604","repositories_listed":1,"syntology":{"n":1,"n_ran":0,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/rethinking-medical-report-generation-disease","title":"Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graph","date":"2023-07-24","arxiv_id":"2307.12526","repositories_listed":1,"syntology":null},{"url":"/paper/multi-modal-pre-training-for-medical-vision","title":"Multi-modal Pre-training for Medical Vision-language Understanding and Generation: An Empirical Study with A New Benchmark","date":"2023-06-10","arxiv_id":"2306.06494","repositories_listed":1,"syntology":null},{"url":"/paper/organ-observation-guided-radiology-report","title":"ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning","date":"2023-06-10","arxiv_id":"2306.06466","repositories_listed":1,"syntology":null}],"syntology_records":6,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}