Browse State-of-the-Art › Medical Report Generation
Medical Report Generation
46 papers with code · 3 benchmarks · 9 datasets archive 2025-07-28
Medical report generation (MRG) is a task which focus on training AI to automatically generate professional report according the input image data. This can help clinicians make faster and more accurate decision since the task itself is both time consuming and error prone even for experienced doctors.
Aggfgg
Deep neural network and transformer based architecture are currently the most popular methods for this certain task, however, when we try to transfer out pre-trained model into this certain domain, their performance always degrade.
The following are some of the reasons why RSG is hard for pre-trained models:
- Language datasets in a particular domain can sometimes be quite different from the large number of datasets available on the Internet
- During the fine-tuning phase, datasets in the medical field are often unevenly distributed
More recently, multi-modal learning and contrastive learning have shown some inspiring results in this field, but it's still challenging and requires further attention.
Here are some additional readings to go deeper on the task:
- On the Automatic Generation of Medical Imaging Reports
https://doi.org/10.48550/arXiv.1711.08195
- A scoping review of transfer learning research on medical image analysis using ImageNet
https://arxiv.org/abs/2004.13175
- A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
https://arxiv.org/abs/2004.12150
(Image credit : Transformers in Medical Imaging: A Survey)
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology 5 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): MIMIC-CXR (5 judged, 0 placed), IU X-Ray (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| MIMIC-CXR (2 rows) | RGRG | Interactive and Explainable Region-guided Radiology Report Generation | code | Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified | Compare |
| HistGen WSI-Report Dataset (1 row) | HistGen | HistGen: Histopathology Report Generation via Local-Global Feature... | code | Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified | Compare |
| IU X-Ray (1 row) | X-RGen | Act Like a Radiologist: Radiology Report Generation across... | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
9 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 110 · with a code link: 46 · where Syntology ran a sample: 8 (7 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (8 of 110 tagged: 7 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 110, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
22 Nov 2017 4 repositories listedTo cope with these challenges, we (1) build a multi-task learning framework which jointly performs the pre- diction of tags and the generation of para- graphs, (2) propose a co-attention mechanism to localize regions…
-
30 Jun 2023 2 repositories listedWe present CausalVLR (Causal Visual-Linguistic Reasoning), an open-source toolbox containing a rich set of state-of-the-art causal relation discovery and causal inference methods for various visual-linguistic reasoning…
-
26 May 2023 2 repositories listedTo address these issues, we propose X-RGen, a radiologist-minded report generation framework across six anatomical regions.
-
16 Mar 2023 2 repositories listedMedical report generation (MRG) is essential for computer-aided diagnosis and medication guidance, which can relieve the heavy burden of radiologists by automatically generating the corresponding medical reports…
-
6 Jun 2020 2 repositories listedFirstly, the regions of primary interest to radiologists are usually located in a small area of the global image, meaning that the remainder parts of the image could be considered as irrelevant noise in the training…
-
23 Mar 2025 1 repository listedRetrieval-augmented generation (RAG) has emerged as a pivotal technique in artificial intelligence (AI), particularly in enhancing the capabilities of large language models (LLMs) by enabling access to external…
-
20 Mar 2025 1 repository listedIn addition, it is applicable to multiple imaging modalities (e. g., X-ray, CT and PET), covering a wide range of applications from basic diagnostics to complex lesion analysis.
-
7 Mar 2025 1 repository listedHowever, existing evaluation metrics primarily assess the accuracy of key medical information coverage in generated reports compared to human-written reports, while overlooking crucial details such as the location and…
-
7 Jan 2025 1 repository listedX-ray image based medical report generation achieves significant progress in recent years with the help of the large language model, however, these models have not fully exploited the effective information in visual…
-
5 Dec 2024 1 repository listedIt significantly outperforms the state-of-the-art reference model by Qiu et al., achieving a METEOR score of 55.
-
19 Nov 2024 1 repository listedUnlike general vision-and-language models trained on diverse, non-specialized datasets, MVLMs are purpose-built for the medical domain, automatically extracting and interpreting critical information from medical images…
-
1 Oct 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)Thus, we conduct a comprehensive benchmarking of existing mainstream X-ray report generation models and large language models (LLMs), on the CheXpert Plus dataset.
-
19 Aug 2024 1 repository listedThey usually adopt a Transformer to extract the visual features of a given X-ray image, and then, feed them into the LLM for text generation.
-
16 Aug 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 8 unverified (of 14 harvested samples) · 14 pointer-only (licence)We established a benchmark for the ECG report generation task and tested our model on multiple baselines.
-
4 Jul 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)Our empirical assessments confirm MiniGPT-Med's superior performance in disease grounding, medical report generation, and VQA benchmarks, representing a significant step towards reducing the gap in assisting radiology…
-
2 Jul 2024 1 repository listedMulti-modal large language models (MLLMs) have been given free rein to explore exciting medical applications with a primary focus on radiology report generation.
-
23 May 2024 1 repository listedThis process allows the text decoder to attend to discriminative features of X-ray images, assimilate historical diagnostic information from similar cases, and understand the examination intention of patients.
-
15 May 2024 1 repository listedIn Stage 1, we introduce factuality-guided contrastive learning for visual representation by maximizing the semantic correspondence between radiographs and corresponding factual descriptions.
-
23 Apr 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)Specifically, we propose a cooperative framework, Generalist-Specialist Collaboration (GSCo), which consists of two stages, namely the construction of GFM and specialists, and collaborative inference on downstream tasks.
-
8 Mar 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 1 pointer-only (licence)Histopathology serves as the gold standard in cancer diagnosis, with clinical reports being vital in interpreting and understanding this process, guiding cancer treatment and patient care.
-
4 Mar 2024 1 repository listedOur paper reviews recent advancements in developing VLMs specialized for healthcare, focusing on models designed for medical report generation and visual question answering (VQA).
-
20 Feb 2024 1 repository listedPrevious research on radiology report generation has made significant progress in terms of increasing the clinical accuracy of generated reports.
-
5 Jan 2024 1 repository listedIn this paper, we propose a parameter efficient framework for fine-tuning MLLMs, specifically validated on medical visual question answering (Med-VQA) and medical report generation (MRG) tasks, using public benchmark…
-
7 Dec 2023 1 repository listedMedical report generation demands automatic creation of coherent and precise descriptions for medical images.
-
4 Nov 2023 1 repository listedThe goal of automatic report generation is to generate a clinically accurate and coherent phrase from a single given X-ray image, which could alleviate the workload of traditional radiology reporting.
-
21 Oct 2023 1 repository listedIt then combines the historical records, spatiotemporal information, and radiographs for report generation, where a disease progression graph and dynamic progression reasoning mechanism are devised to accurately select…
-
24 Aug 2023 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)To address these challenges, we propose diagnosis-driven prompts for medical report generation (PromptMRG), a novel framework that aims to improve the diagnostic accuracy of MRG with the guidance of diagnosis-aware…
-
24 Jul 2023 1 repository listedKnowledge Graph (KG) plays a crucial role in Medical Report Generation (MRG) because it reveals the relations among diseases and thus can be utilized to guide the generation process.
-
10 Jun 2023 1 repository listedWith the availability of large-scale, comprehensive, and general-purpose vision-language (VL) datasets such as MSCOCO, vision-language pre-training (VLP) has become an active area of research and proven to be effective…
-
10 Jun 2023 1 repository listedThis paper explores the task of radiology report generation, which aims at generating free-text descriptions for a set of radiographs.
-
11 May 2023 1 repository listed
-
17 Apr 2023 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 10 harvested samples)
-
18 Mar 2023 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 9 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 3 pointer-only (licence)
-
16 Nov 2022 1 repository listed
-
12 Nov 2022 1 repository listed
-
15 Sep 2022 1 repository listed
-
19 Apr 2022 1 repository listed
-
24 Jan 2022 1 repository listed
-
1 Nov 2021 1 repository listed
-
25 Sep 2021 1 repository listed
-
19 Aug 2021 1 repository listed
-
26 Mar 2021 1 repository listed
-
20 Feb 2021 1 repository listed Syntology official: harvested, nothing ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
18 Nov 2020 1 repository listed
-
1 Nov 2020 1 repository listed
-
31 Oct 2019 1 repository listed
-
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun 2025 0 repositories listed
-
MRGAgents: A Multi-Agent Framework for Improved Medical Report Generation with Med-LVLMs24 May 2025 0 repositories listed
-
Towards a HIPAA Compliant Agentic AI System in Healthcare24 Apr 2025 0 repositories listed
-
LVMed-R2: Perception and Reflection-driven Complex Reasoning for Medical Report Generation2 Apr 2025 0 repositories listed
-
Image-to-Text for Medical Reports Using Adaptive Co-Attention and Triple-LSTM Module24 Mar 2025 0 repositories listed
-
MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations2 Mar 2025 0 repositories listed
-
PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation14 Feb 2025 0 repositories listed
-
From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine13 Feb 2025 0 repositories listed
-
DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching19 Dec 2024 0 repositories listed
-
FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models27 Nov 2024 0 repositories listed
-
The Potential of LLMs in Medical Education: Generating Questions and Answers for Qualification Exams31 Oct 2024 0 repositories listed
-
Large Language Model Benchmarks in Medical Tasks28 Oct 2024 0 repositories listed
-
Image-aware Evaluation of Generated Medical Reports22 Oct 2024 0 repositories listed
-
Resource-Efficient Medical Report Generation using Large Language Models21 Oct 2024 0 repositories listed
-
Retrieval Instead of Fine-tuning: A Retrieval-based Parameter Ensemble for Zero-shot Learning13 Oct 2024 0 repositories listed
-
ViT3D Alignment of LLaMA3: 3D Medical Image Report Generation11 Oct 2024 0 repositories listed
-
FODA-PG for Enhanced Medical Imaging Narrative Generation: Adaptive Differentiation of Normal and Abnormal Attributes6 Sep 2024 0 repositories listed
-
Medical Report Generation Is A Multi-label Classification Problem30 Aug 2024 0 repositories listed
-
M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretation29 Aug 2024 0 repositories listed
-
Automatic Medical Report Generation: Methods and Applications26 Aug 2024 0 repositories listed
-
Automated Retinal Image Analysis and Medical Report Generation through Deep Learning14 Aug 2024 0 repositories listed
-
A Labeled Ophthalmic Ultrasound Dataset with Medical Report Generation Based on Cross-modal Deep Learning26 Jul 2024 0 repositories listed
-
MedRAT: Unpaired Medical Report Generation via Auxiliary Tasks4 Jul 2024 0 repositories listed
-
A Survey on Trustworthiness in Foundation Models for Medical Image Analysis3 Jul 2024 0 repositories listed
-
CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation17 Jun 2024 0 repositories listed
-
Topicwise Separable Sentence Retrieval for Medical Report Generation7 May 2024 0 repositories listed
-
Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM17 Apr 2024 0 repositories listed
-
Dia-LLaMA: Towards Large Language Model-driven CT Report Generation25 Mar 2024 0 repositories listed
-
MedCycle: Unpaired Medical Report Generation via Cycle-Consistency20 Mar 2024 0 repositories listed
-
Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation25 Jan 2024 0 repositories listed
-
Dynamic Traceback Learning for Medical Report Generation24 Jan 2024 0 repositories listed
-
Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning26 Dec 2023 0 repositories listed
-
C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network9 Oct 2023 0 repositories listed
-
IIHT: Medical Report Generation with Image-to-Indicator Hierarchical Transformer10 Aug 2023 0 repositories listed
-
Customizing General-Purpose Foundation Models for Medical Report Generation9 Jun 2023 0 repositories listed
-
Boosting Radiology Report Generation by Infusing Comparison Prior8 May 2023 0 repositories listed
-
MvCo-DoT:Multi-View Contrastive Domain Transfer Network for Medical Report Generation15 Apr 2023 0 repositories listed
-
Cyclic Generative Adversarial Networks With Congruent Image-Report Generation For Explainable Medical Image Analysis16 Nov 2022 0 repositories listed
-
Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penalty14 Oct 2022 0 repositories listed
-
Cross-modal Contrastive Attention Model for Medical Report Generation1 Oct 2022 0 repositories listed
-
DeltaNet: Conditional Medical Report Generation for COVID-19 Diagnosis1 Oct 2022 0 repositories listed
-
JPG - Jointly Learn to Align: Automated Disease Prediction and Radiology Report Generation1 Oct 2022 0 repositories listed
-
Representative Image Feature Extraction via Contrastive Learning Pretraining for Chest X-ray Report Generation4 Sep 2022 0 repositories listed
-
A Medical Semantic-Assisted Transformer for Radiographic Report Generation22 Aug 2022 0 repositories listed
-
Competence-based Multimodal Curriculum Learning for Medical Report Generation24 Jun 2022 0 repositories listed
-
A Self-Guided Framework for Radiology Report Generation19 Jun 2022 0 repositories listed
-
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation4 Jun 2022 0 repositories listed
-
AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation18 Mar 2022 0 repositories listed
-
Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation12 Mar 2022 0 repositories listed
-
Brain Cancer Survival Prediction on Treatment-na ive MRI using Deep Anchor Attention Learning with Vision Transformer3 Feb 2022 0 repositories listed
-
Towards Building Automatic Medical Consultation System: Framework, Task and Dataset16 Nov 2021 0 repositories listed
-
Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation8 Nov 2021 0 repositories listed
-
Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning11 Aug 2021 0 repositories listed
-
A Self-Boosting Framework for Automated Radiographic Report Generation19 Jun 2021 0 repositories listed
Syntology lines on 9 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections