{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fg-cxr-a-radiologist-aligned-gaze-dataset-for","title":"FG-CXR: A Radiologist-Aligned Gaze Dataset for Enhancing Interpretability in Chest X-Ray Report Generation","arxiv_id":"2411.15413","date":"2024-11-23","proceeding":null,"authors":["Trong Thang Pham","Ngoc-Vuong Ho","Nhat-Tan Bui","Thinh Phan","Patel Brijesh","Donald Adjeroh","Gianfranco Doretto","Anh Nguyen","Carol C. Wu","Hien Nguyen","Ngan Le"],"abstract":"Developing an interpretable system for generating reports in chest X-ray (CXR) analysis is becoming increasingly crucial in Computer-aided Diagnosis (CAD) systems, enabling radiologists to comprehend the decisions made by these systems. Despite the growth of diverse datasets and methods focusing on report generation, there remains a notable gap in how closely these models' generated reports align with the interpretations of real radiologists. In this study, we tackle this challenge by initially introducing Fine-Grained CXR (FG-CXR) dataset, which provides fine-grained paired information between the captions generated by radiologists and the corresponding gaze attention heatmaps for each anatomy. Unlike existing datasets that include a raw sequence of gaze alongside a report, with significant misalignment between gaze location and report content, our FG-CXR dataset offers a more grained alignment between gaze attention and diagnosis transcript. Furthermore, our analysis reveals that simply applying black-box image captioning methods to generate reports cannot adequately explain which information in CXR is utilized and how long needs to attend to accurately generate reports. Consequently, we propose a novel explainable radiologist's attention generator network (Gen-XAI) that mimics the diagnosis process of radiologists, explicitly constraining its output to closely align with both radiologist's gaze attention and transcript. Finally, we perform extensive experiments to illustrate the effectiveness of our method. Our datasets and checkpoint is available at https://github.com/UARK-AICV/FG-CXR.","url_abs":"https://arxiv.org/abs/2411.15413v1","url_pdf":"https://arxiv.org/pdf/2411.15413v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fg-cxr-a-radiologist-aligned-gaze-dataset-for","repo_url":"https://github.com/uark-aicv/fg-cxr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"anatomy","task_name":"Anatomy"},{"task_slug":"image-captioning","task_name":"Image Captioning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.15413","atlas_url":"https://app.syntology.ai/?focus=2411.15413","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.15413"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/uark-aicv/fg-cxr","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"02cc8fd057a0e20d","entry":"checkpoint_payload","repo":"uark-aicv/fg-cxr","repo_kind":"official","path":"fgcxr/train_report.py","file_url":"https://github.com/uark-aicv/fg-cxr/blob/HEAD/fgcxr/train_report.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02cc8fd057a0e20d"}},{"code_sha256_prefix":"48e2446cf9649d09","entry":"checkpoint_payload","repo":"uark-aicv/fg-cxr","repo_kind":"official","path":"fgcxr/train_heatmap.py","file_url":"https://github.com/uark-aicv/fg-cxr/blob/HEAD/fgcxr/train_heatmap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48e2446cf9649d09"}},{"code_sha256_prefix":"d7dc8571cd47ac18","entry":"tokenize_reports","repo":"uark-aicv/fg-cxr","repo_kind":"official","path":"fgcxr/train_report.py","file_url":"https://github.com/uark-aicv/fg-cxr/blob/HEAD/fgcxr/train_report.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7dc8571cd47ac18"}},{"code_sha256_prefix":"814a03caeec6a19e","entry":"validate","repo":"uark-aicv/fg-cxr","repo_kind":"official","path":"fgcxr/train_heatmap.py","file_url":"https://github.com/uark-aicv/fg-cxr/blob/HEAD/fgcxr/train_heatmap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"814a03caeec6a19e"}},{"code_sha256_prefix":"e1ade8830ffcba3e","entry":"validate","repo":"uark-aicv/fg-cxr","repo_kind":"official","path":"fgcxr/train_report.py","file_url":"https://github.com/uark-aicv/fg-cxr/blob/HEAD/fgcxr/train_report.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1ade8830ffcba3e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}