{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/structural-entities-extraction-and-patient","title":"Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation","arxiv_id":"2405.14905","date":"2024-05-23","proceeding":null,"authors":["Kang Liu","Zhuoqi Ma","Xiaolu Kang","Zhusi Zhong","Zhicheng Jiao","Grayson Baird","Harrison Bai","Qiguang Miao"],"abstract":"The automated generation of imaging reports proves invaluable in alleviating the workload of radiologists. A clinically applicable reports generation algorithm should demonstrate its effectiveness in producing reports that accurately describe radiology findings and attend to patient-specific indications. In this paper, we introduce a novel method, \\textbf{S}tructural \\textbf{E}ntities extraction and patient indications \\textbf{I}ncorporation (SEI) for chest X-ray report generation. Specifically, we employ a structural entities extraction (SEE) approach to eliminate presentation-style vocabulary in reports and improve the quality of factual entity sequences. This reduces the noise in the following cross-modal alignment module by aligning X-ray images with factual entity sequences in reports, thereby enhancing the precision of cross-modal alignment and further aiding the model in gradient-free retrieval of similar historical cases. Subsequently, we propose a cross-modal fusion network to integrate information from X-ray images, similar historical cases, and patient-specific indications. This process allows the text decoder to attend to discriminative features of X-ray images, assimilate historical diagnostic information from similar cases, and understand the examination intention of patients. This, in turn, assists in triggering the text decoder to produce high-quality reports. Experiments conducted on MIMIC-CXR validate the superiority of SEI over state-of-the-art approaches on both natural language generation and clinical efficacy metrics.","url_abs":"https://arxiv.org/abs/2405.14905v1","url_pdf":"https://arxiv.org/pdf/2405.14905v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"structural-entities-extraction-and-patient","repo_url":"https://github.com/mk-runner/sei","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"medical-report-generation","task_name":"Medical Report Generation"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"cross-modal-alignment","task_name":"cross-modal alignment"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/medical-report-generation-on-mimic-cxr","task":"Medical Report Generation","dataset":"MIMIC-CXR","model":"SEI-1","rank_in_archive_order":2,"of":2,"metrics":{"BLEU-2":"0.247","BLEU-4":"0.135","Example-F1-14":"0.460","F1 RadGraph":"0.249","METEOR":"0.158","Micro-F1-5":"0.542","ROUGE-L":"0.299"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.14905","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}