{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/evidence-is-all-you-need-ordering-imaging","title":"Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness Criteria","arxiv_id":"2409.19177","date":"2024-09-27","proceeding":null,"authors":["Michael S. Yao","Allison Chae","Charles E. Kahn Jr.","Walter R. Witschey","James C. Gee","Hersh Sagreiya","Osbert Bastani"],"abstract":"Diagnostic imaging studies are an increasingly important component of the workup and management of acutely presenting patients. However, ordering appropriate imaging studies according to evidence-based medical guidelines is a challenging task with a high degree of variability between healthcare providers. To address this issue, recent work has investigated if generative AI and large language models can be leveraged to help clinicians order relevant imaging studies for patients. However, it is challenging to ensure that these tools are correctly aligned with medical guidelines, such as the American College of Radiology's Appropriateness Criteria (ACR AC). In this study, we introduce a framework to intelligently leverage language models by recommending imaging studies for patient cases that are aligned with evidence-based guidelines. We make available a novel dataset of patient \"one-liner\" scenarios to power our experiments, and optimize state-of-the-art language models to achieve an accuracy on par with clinicians in image ordering. Finally, we demonstrate that our language model-based pipeline can be used as intelligent assistants by clinicians to support image ordering workflows and improve the accuracy of imaging study ordering according to the ACR AC. Our work demonstrates and validates a strategy to leverage AI-based software to improve trustworthy clinical decision making in alignment with expert evidence-based guidelines.","url_abs":"https://arxiv.org/abs/2409.19177v2","url_pdf":"https://arxiv.org/pdf/2409.19177v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evidence-is-all-you-need-ordering-imaging","repo_url":"https://github.com/michael-s-yao/radGPT","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[{"method_slug":null,"method_name":"American"}],"datasets_introduced":[{"slug":"acr-appropriateness-criteria-corpus","name":"ACR Appropriateness Criteria Corpus","full_name":""},{"slug":"radcases","name":"RadCases","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.19177","atlas_url":"https://app.syntology.ai/?focus=2409.19177","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.19177"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michael-s-yao/radGPT","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f69f9d0af8972175","entry":"score","repo":"michael-s-yao/radGPT","repo_kind":"official","path":"radgpt/utils.py","file_url":"https://github.com/michael-s-yao/radGPT/blob/HEAD/radgpt/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f69f9d0af8972175"}},{"code_sha256_prefix":"fd34dbdf7f23af0f","entry":"split_into_sentences","repo":"michael-s-yao/radGPT","repo_kind":"official","path":"radgpt/utils.py","file_url":"https://github.com/michael-s-yao/radGPT/blob/HEAD/radgpt/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd34dbdf7f23af0f"}},{"code_sha256_prefix":"cb1abb8d08bd1c2a","entry":"build_finetuning_dataset","repo":"michael-s-yao/radGPT","repo_kind":"official","path":"radgpt/finetuning.py","file_url":"https://github.com/michael-s-yao/radGPT/blob/HEAD/radgpt/finetuning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb1abb8d08bd1c2a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}