{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ehrmamba-towards-generalizable-and-scalable","title":"EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records","arxiv_id":"2405.14567","date":"2024-05-23","proceeding":null,"authors":["Adibvafa Fallahpour","Mahshid Alinoori","Wenqian Ye","Xu Cao","Arash Afkanpour","Amrit Krishnan"],"abstract":"Transformers have significantly advanced the modeling of Electronic Health Records (EHR), yet their deployment in real-world healthcare is limited by several key challenges. Firstly, the quadratic computational cost and insufficient context length of these models hinder hospitals' ability in processing the extensive medical histories typical in EHR data. Additionally, existing models employ separate finetuning for each clinical task, complicating maintenance in healthcare environments. Moreover, these models focus exclusively on either clinical prediction or EHR forecasting, lacking proficiency in both tasks. To overcome these limitations, we introduce EHRMamba, a robust foundation model built on the Mamba architecture. EHRMamba can process sequences up to 300% longer than previous models due to its linear computational cost. We also introduce a novel approach to Multitask Prompted Finetuning (MPF) for EHR data, which enables EHRMamba to simultaneously learn multiple clinical tasks in a single finetuning phase, significantly enhancing deployment and cross-task generalization. Furthermore, our model leverages the HL7 FHIR data standard to simplify integration into existing hospital systems. Alongside EHRMamba, we open-source Odyssey, a toolkit designed to support the development and deployment of EHR foundation models, with an emphasis on data standardization and interpretability. Our evaluations on the MIMIC-IV dataset demonstrate that EHRMamba advances state-of-the-art performance across 6 major clinical tasks and excels in EHR forecasting, marking a significant leap forward in the field.","url_abs":"https://arxiv.org/abs/2405.14567v3","url_pdf":"https://arxiv.org/pdf/2405.14567v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ehrmamba-towards-generalizable-and-scalable","repo_url":"https://github.com/VectorInstitute/odyssey","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"mamba","task_name":"Mamba"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2405.14567","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14567"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VectorInstitute/odyssey","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":14},"by_repo_kind":{"official":{"samples":14,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"652cb2ddc9a741b5","entry":"annealed_alpha","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/concept_bottleneck.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/concept_bottleneck.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"652cb2ddc9a741b5"}},{"code_sha256_prefix":"9cc15076d0a08f28","entry":"best_scorer_per_cell","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/inference/model_comparison.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/inference/model_comparison.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9cc15076d0a08f28"}},{"code_sha256_prefix":"2407e51f7377ab2c","entry":"censored_hazard_log_likelihood","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/time_to_event.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/time_to_event.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2407e51f7377ab2c"}},{"code_sha256_prefix":"b040d3186985f9cd","entry":"concept_loss","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/concept_bottleneck.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/concept_bottleneck.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b040d3186985f9cd"}},{"code_sha256_prefix":"20e5b21ec781dd3a","entry":"gap_to_bin","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/time_to_event.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/time_to_event.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"20e5b21ec781dd3a"}},{"code_sha256_prefix":"fa134ddfdde7b9ed","entry":"hazard_log_likelihood","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/time_to_event.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/time_to_event.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa134ddfdde7b9ed"}},{"code_sha256_prefix":"1257aeb0e0ca5e64","entry":"intervention_apply_mask","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/concept_bottleneck.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/concept_bottleneck.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1257aeb0e0ca5e64"}},{"code_sha256_prefix":"37d5d57f4239c77d","entry":"middle_layer","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/injection.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/injection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37d5d57f4239c77d"}},{"code_sha256_prefix":"ab16e79712613f58","entry":"pinball_loss","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/value_head.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/value_head.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ab16e79712613f58"}},{"code_sha256_prefix":"20f21b3aa186f2d8","entry":"select_diverse_cases","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/inference/case_study.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/inference/case_study.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"20f21b3aa186f2d8"}},{"code_sha256_prefix":"04ebfb65c3483398","entry":"stream_injection","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/injection.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/injection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"04ebfb65c3483398"}},{"code_sha256_prefix":"816376e6520ac858","entry":"value_features","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/embeddings.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"816376e6520ac858"}},{"code_sha256_prefix":"0a2999efd7fc3d1b","entry":"value_quantile_loss","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/value_head.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/value_head.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a2999efd7fc3d1b"}},{"code_sha256_prefix":"ab2ebe14caf34af0","entry":"value_target_valid_mask","repo":"VectorInstitute/odyssey","repo_kind":"official","path":"odyssey/models/value_head.py","file_url":"https://github.com/VectorInstitute/odyssey/blob/HEAD/odyssey/models/value_head.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ab2ebe14caf34af0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}