{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-comprehensive-benchmark-for-covid-19","title":"A Comprehensive Benchmark for COVID-19 Predictive Modeling Using Electronic Health Records in Intensive Care","arxiv_id":"2209.07805","date":"2022-09-16","proceeding":null,"authors":["Junyi Gao","Yinghao Zhu","Wenqing Wang","Yasha Wang","Wen Tang","Ewen M. Harrison","Liantao Ma"],"abstract":"The COVID-19 pandemic has posed a heavy burden to the healthcare system worldwide and caused huge social disruption and economic loss. Many deep learning models have been proposed to conduct clinical predictive tasks such as mortality prediction for COVID-19 patients in intensive care units using Electronic Health Record (EHR) data. Despite their initial success in certain clinical applications, there is currently a lack of benchmarking results to achieve a fair comparison so that we can select the optimal model for clinical use. Furthermore, there is a discrepancy between the formulation of traditional prediction tasks and real-world clinical practice in intensive care. To fill these gaps, we propose two clinical prediction tasks, Outcome-specific length-of-stay prediction and Early mortality prediction for COVID-19 patients in intensive care units. The two tasks are adapted from the naive length-of-stay and mortality prediction tasks to accommodate the clinical practice for COVID-19 patients. We propose fair, detailed, open-source data-preprocessing pipelines and evaluate 17 state-of-the-art predictive models on two tasks, including 5 machine learning models, 6 basic deep learning models and 6 deep learning predictive models specifically designed for EHR data. We provide benchmarking results using data from two real-world COVID-19 EHR datasets. One dataset is publicly available without needing any inquiry and another dataset can be accessed on request. We provide fair, reproducible benchmarking results for two tasks. We deploy all experiment results and models on an online platform. We also allow clinicians and researchers to upload their data to the platform and get quick prediction results using our trained models. We hope our efforts can further facilitate deep learning and machine learning research for COVID-19 predictive modeling.","url_abs":"https://arxiv.org/abs/2209.07805v4","url_pdf":"https://arxiv.org/pdf/2209.07805v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-comprehensive-benchmark-for-covid-19","repo_url":"https://github.com/yhzhu99/covid-ehr-benchmarks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-comprehensive-benchmark-for-covid-19","repo_url":"https://github.com/yhzhu99/pyehr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"a-comprehensive-benchmark-for-covid-19","repo_url":"https://github.com/yhzhu99/pyehr-playground","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"length-of-stay-prediction","task_name":"Length-of-Stay prediction"},{"task_slug":"mortality-prediction","task_name":"Mortality Prediction"},{"task_slug":"prediction","task_name":"Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2209.07805","atlas_url":"https://app.syntology.ai/?focus=2209.07805","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.07805"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yhzhu99/pyehr","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yhzhu99/pyehr-playground","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yhzhu99/covid-ehr-benchmarks","reach":null}],"summary":{"ran_draft_wrong":2,"ran":5,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"b7cd239976f57e30","entry":"calculate_los_statistics","repo":"yhzhu99/covid-ehr-benchmarks","repo_kind":"official","path":"app/apis/dl_los_pipeline.py","file_url":"https://github.com/yhzhu99/covid-ehr-benchmarks/blob/HEAD/app/apis/dl_los_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b7cd239976f57e30"}},{"code_sha256_prefix":"38f7f8ad3286b936","entry":"calculate_los_statistics","repo":"yhzhu99/covid-ehr-benchmarks","repo_kind":"official","path":"app/apis/ml_los_pipeline.py","file_url":"https://github.com/yhzhu99/covid-ehr-benchmarks/blob/HEAD/app/apis/ml_los_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"GPL-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"38f7f8ad3286b936"}},{"code_sha256_prefix":"efdc8fa2e3f3e60f","entry":"compute_codes","repo":"yhzhu99/pyehr","repo_kind":"official","path":"models/grasp.py","file_url":"https://github.com/yhzhu99/pyehr/blob/HEAD/models/grasp.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efdc8fa2e3f3e60f"}},{"code_sha256_prefix":"093b34abc64726bc","entry":"get_multitask_loss","repo":"yhzhu99/pyehr","repo_kind":"official","path":"losses/multitask_loss.py","file_url":"https://github.com/yhzhu99/pyehr/blob/HEAD/losses/multitask_loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"093b34abc64726bc"}},{"code_sha256_prefix":"b2ce212b04b4c0ac","entry":"get_time_aware_loss","repo":"yhzhu99/pyehr","repo_kind":"official","path":"losses/time_aware_loss.py","file_url":"https://github.com/yhzhu99/pyehr/blob/HEAD/losses/time_aware_loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2ce212b04b4c0ac"}},{"code_sha256_prefix":"ecf8d0b932ea3b5e","entry":"random_init","repo":"yhzhu99/pyehr","repo_kind":"official","path":"models/grasp.py","file_url":"https://github.com/yhzhu99/pyehr/blob/HEAD/models/grasp.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ecf8d0b932ea3b5e"}},{"code_sha256_prefix":"628681867a4d6000","entry":"update_centers","repo":"yhzhu99/pyehr","repo_kind":"official","path":"models/grasp.py","file_url":"https://github.com/yhzhu99/pyehr/blob/HEAD/models/grasp.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"628681867a4d6000"}},{"code_sha256_prefix":"e3ce9bc01495f660","entry":"train_epoch","repo":"yhzhu99/covid-ehr-benchmarks","repo_kind":"official","path":"app/apis/dl_los_pipeline.py","file_url":"https://github.com/yhzhu99/covid-ehr-benchmarks/blob/HEAD/app/apis/dl_los_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e3ce9bc01495f660"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}