{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exbehrt-extended-transformer-for-electronic","title":"ExBEHRT: Extended Transformer for Electronic Health Records to Predict Disease Subtypes & Progressions","arxiv_id":"2303.12364","date":"2023-03-22","proceeding":null,"authors":["Maurice Rupp","Oriane Peter","Thirupathi Pattipaka"],"abstract":"In this study, we introduce ExBEHRT, an extended version of BEHRT (BERT applied to electronic health records), and apply different algorithms to interpret its results. While BEHRT considers only diagnoses and patient age, we extend the feature space to several multimodal records, namely demographics, clinical characteristics, vital signs, smoking status, diagnoses, procedures, medications, and laboratory tests, by applying a novel method to unify the frequencies and temporal dimensions of the different features. We show that additional features significantly improve model performance for various downstream tasks in different diseases. To ensure robustness, we interpret model predictions using an adaptation of expected gradients, which has not been previously applied to transformers with EHR data and provides more granular interpretations than previous approaches such as feature and token importances. Furthermore, by clustering the model representations of oncology patients, we show that the model has an implicit understanding of the disease and is able to classify patients with the same cancer type into different risk groups. Given the additional features and interpretability, ExBEHRT can help make informed decisions about disease trajectories, diagnoses, and risk factors of various diseases.","url_abs":"https://arxiv.org/abs/2303.12364v3","url_pdf":"https://arxiv.org/pdf/2303.12364v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exbehrt-extended-transformer-for-electronic","repo_url":"https://github.com/ZhiGroup/Med-BERT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.12364","atlas_url":"https://app.syntology.ai/?focus=2303.12364","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.12364"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/deepmedicine/BEHRT","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZhiGroup/Med-BERT","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7d49de96ff7f46a4","entry":"age_vocab","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"model/utils.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/model/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d49de96ff7f46a4"}},{"code_sha256_prefix":"9fd9d3038a5a60ca","entry":"code2index","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"dataLoader/utils.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/dataLoader/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fd9d3038a5a60ca"}},{"code_sha256_prefix":"0398a15357fa6b92","entry":"index_seg","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"dataLoader/utils.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/dataLoader/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0398a15357fa6b92"}},{"code_sha256_prefix":"28c0cc0d046b452f","entry":"load_model","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"common/pytorch.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/common/pytorch.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"28c0cc0d046b452f"}},{"code_sha256_prefix":"983e282e8dca4cb1","entry":"load_obj","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"common/common.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/common/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"983e282e8dca4cb1"}},{"code_sha256_prefix":"d35ee78e76ea22cc","entry":"random_mask","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"dataLoader/utils.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/dataLoader/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d35ee78e76ea22cc"}},{"code_sha256_prefix":"a2d0c3cd0740e312","entry":"read_csv","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"common/spark.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/common/spark.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2d0c3cd0740e312"}},{"code_sha256_prefix":"40dad3a05f75fd73","entry":"read_parquet","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"common/spark.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/common/spark.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"40dad3a05f75fd73"}},{"code_sha256_prefix":"f27f6d5ef718adaf","entry":"read_txt","repo":"deepmedicine/BEHRT","repo_kind":"found_in_text","path":"common/spark.py","file_url":"https://github.com/deepmedicine/BEHRT/blob/HEAD/common/spark.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f27f6d5ef718adaf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}