{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/radgpt-constructing-3d-image-text-tumor","title":"RadGPT: Constructing 3D Image-Text Tumor Datasets","arxiv_id":"2501.04678","date":"2025-01-08","proceeding":null,"authors":["Pedro R. A. S. Bassi","Mehmet Can Yavuz","Kang Wang","Xiaoxi Chen","Wenxuan Li","Sergio Decherchi","Andrea Cavalli","Yang Yang","Alan Yuille","Zongwei Zhou"],"abstract":"With over 85 million CT scans performed annually in the United States, creating tumor-related reports is a challenging and time-consuming task for radiologists. To address this need, we present RadGPT, an Anatomy-Aware Vision-Language AI Agent for generating detailed reports from CT scans. RadGPT first segments tumors, including benign cysts and malignant tumors, and their surrounding anatomical structures, then transforms this information into both structured reports and narrative reports. These reports provide tumor size, shape, location, attenuation, volume, and interactions with surrounding blood vessels and organs. Extensive evaluation on unseen hospitals shows that RadGPT can produce accurate reports, with high sensitivity/specificity for small tumor (<2 cm) detection: 80/73% for liver tumors, 92/78% for kidney tumors, and 77/77% for pancreatic tumors. For large tumors, sensitivity ranges from 89% to 97%. The results significantly surpass the state-of-the-art in abdominal CT report generation. RadGPT generated reports for 17 public datasets. Through radiologist review and refinement, we have ensured the reports' accuracy, and created the first publicly available image-text 3D medical dataset, comprising over 1.8 million text tokens and 2.7 million images from 9,262 CT scans, including 2,947 tumor scans/reports of 8,562 tumor instances. Our reports can: (1) localize tumors in eight liver sub-segments and three pancreatic sub-segments annotated per-voxel; (2) determine pancreatic tumor stage (T1-T4) in 260 reports; and (3) present individual analyses of multiple tumors--rare in human-made reports. Importantly, 948 of the reports are for early-stage tumors.","url_abs":"https://arxiv.org/abs/2501.04678v1","url_pdf":"https://arxiv.org/pdf/2501.04678v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"radgpt-constructing-3d-image-text-tumor","repo_url":"https://github.com/mrgiovanni/radgpt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"ai-agent","task_name":"AI Agent"},{"task_slug":"anatomy","task_name":"Anatomy"},{"task_slug":"sensitivity","task_name":"Sensitivity"},{"task_slug":"specificity","task_name":"Specificity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.04678","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.04678"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mrgiovanni/radgpt","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/MrGiovanni/RadGPT","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"d7c697be98845662","entry":"CreateConversation","repo":"MrGiovanni/RadGPT","repo_kind":"official","path":"evaluate_reports/RadGPT.py","file_url":"https://github.com/MrGiovanni/RadGPT/blob/HEAD/evaluate_reports/RadGPT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d7c697be98845662"}},{"code_sha256_prefix":"1a4f0030d3bb4c47","entry":"request_API","repo":"MrGiovanni/RadGPT","repo_kind":"official","path":"evaluate_reports/RadGPT.py","file_url":"https://github.com/MrGiovanni/RadGPT/blob/HEAD/evaluate_reports/RadGPT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1a4f0030d3bb4c47"}},{"code_sha256_prefix":"287a323e94178571","entry":"InitializeOpenAIClient","repo":"MrGiovanni/RadGPT","repo_kind":"official","path":"evaluate_reports/RadGPT.py","file_url":"https://github.com/MrGiovanni/RadGPT/blob/HEAD/evaluate_reports/RadGPT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"287a323e94178571"}},{"code_sha256_prefix":"cd8949a713a63d5c","entry":"SendMessageAPI","repo":"MrGiovanni/RadGPT","repo_kind":"official","path":"evaluate_reports/RadGPT.py","file_url":"https://github.com/MrGiovanni/RadGPT/blob/HEAD/evaluate_reports/RadGPT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"cd8949a713a63d5c"}},{"code_sha256_prefix":"279bac808402a3e0","entry":"run_model","repo":"MrGiovanni/RadGPT","repo_kind":"official","path":"evaluate_reports/RadGPT.py","file_url":"https://github.com/MrGiovanni/RadGPT/blob/HEAD/evaluate_reports/RadGPT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"279bac808402a3e0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}