{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-automatic-generation-of-medical","title":"On the Automatic Generation of Medical Imaging Reports","arxiv_id":"1711.08195","date":"2017-11-22","proceeding":"ACL 2018 7","authors":["Baoyu Jing","Pengtao Xie","Eric Xing"],"abstract":"Medical imaging is widely used in clinical practice for diagnosis and\ntreatment. Report-writing can be error-prone for unexperienced physicians, and\ntime- consuming and tedious for experienced physicians. To address these\nissues, we study the automatic generation of medical imaging reports. This task\npresents several challenges. First, a complete report contains multiple\nheterogeneous forms of information, including findings and tags. Second,\nabnormal regions in medical images are difficult to identify. Third, the re-\nports are typically long, containing multiple sentences. To cope with these\nchallenges, we (1) build a multi-task learning framework which jointly performs\nthe pre- diction of tags and the generation of para- graphs, (2) propose a\nco-attention mechanism to localize regions containing abnormalities and\ngenerate narrations for them, (3) develop a hierarchical LSTM model to generate\nlong paragraphs. We demonstrate the effectiveness of the proposed methods on\ntwo publicly available datasets.","url_abs":"http://arxiv.org/abs/1711.08195v3","url_pdf":"http://arxiv.org/pdf/1711.08195v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-automatic-generation-of-medical","repo_url":"https://github.com/Danyache/skoltech_image_cap","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"on-the-automatic-generation-of-medical","repo_url":"https://github.com/Pillercottrer/radcap_project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"on-the-automatic-generation-of-medical","repo_url":"https://github.com/ZexinYan/Medical-Report-Generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"on-the-automatic-generation-of-medical","repo_url":"https://github.com/tantheta01/AGMIR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"medical-report-generation","task_name":"Medical Report Generation"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1711.08195","atlas_url":"https://app.syntology.ai/?focus=1711.08195","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}