{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gram-graph-based-attention-model-for","title":"GRAM: Graph-based Attention Model for Healthcare Representation Learning","arxiv_id":"1611.07012","date":"2016-11-21","proceeding":null,"authors":["Edward Choi","Mohammad Taha Bahadori","Le Song","Walter F. Stewart","Jimeng Sun"],"abstract":"Deep learning methods exhibit promising performance for predictive modeling\nin healthcare, but two important challenges remain: -Data insufficiency:Often\nin healthcare predictive modeling, the sample size is insufficient for deep\nlearning methods to achieve satisfactory results. -Interpretation:The\nrepresentations learned by deep learning methods should align with medical\nknowledge. To address these challenges, we propose a GRaph-based Attention\nModel, GRAM that supplements electronic health records (EHR) with hierarchical\ninformation inherent to medical ontologies. Based on the data volume and the\nontology structure, GRAM represents a medical concept as a combination of its\nancestors in the ontology via an attention mechanism. We compared predictive\nperformance (i.e. accuracy, data needs, interpretability) of GRAM to various\nmethods including the recurrent neural network (RNN) in two sequential\ndiagnoses prediction tasks and one heart failure prediction task. Compared to\nthe basic RNN, GRAM achieved 10% higher accuracy for predicting diseases rarely\nobserved in the training data and 3% improved area under the ROC curve for\npredicting heart failure using an order of magnitude less training data.\nAdditionally, unlike other methods, the medical concept representations learned\nby GRAM are well aligned with the medical ontology. Finally, GRAM exhibits\nintuitive attention behaviors by adaptively generalizing to higher level\nconcepts when facing data insufficiency at the lower level concepts.","url_abs":"http://arxiv.org/abs/1611.07012v3","url_pdf":"http://arxiv.org/pdf/1611.07012v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gram-graph-based-attention-model-for","repo_url":"https://github.com/mp2893/gram","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.07012","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}