{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-multifaceted-benchmarking-of-synthetic","title":"A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Models","arxiv_id":"2208.01230","date":"2022-08-02","proceeding":null,"authors":["Chao Yan","Yao Yan","Zhiyu Wan","Ziqi Zhang","Larsson Omberg","Justin Guinney","Sean D. Mooney","Bradley A. Malin"],"abstract":"Synthetic health data have the potential to mitigate privacy concerns when sharing data to support biomedical research and the development of innovative healthcare applications. Modern approaches for data generation based on machine learning, generative adversarial networks (GAN) methods in particular, continue to evolve and demonstrate remarkable potential. Yet there is a lack of a systematic assessment framework to benchmark methods as they emerge and determine which methods are most appropriate for which use cases. In this work, we introduce a generalizable benchmarking framework to appraise key characteristics of synthetic health data with respect to utility and privacy metrics. We apply the framework to evaluate synthetic data generation methods for electronic health records (EHRs) data from two large academic medical centers with respect to several use cases. The results illustrate that there is a utility-privacy tradeoff for sharing synthetic EHR data. The results further indicate that no method is unequivocally the best on all criteria in each use case, which makes it evident why synthetic data generation methods need to be assessed in context.","url_abs":"https://arxiv.org/abs/2208.01230v1","url_pdf":"https://arxiv.org/pdf/2208.01230v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-multifaceted-benchmarking-of-synthetic","repo_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"synthetic-data-generation","task_name":"Synthetic Data Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2208.01230","atlas_url":"https://app.syntology.ai/?focus=2208.01230","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2208.01230"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yy6linda/synthetic-ehr-benchmarking","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":11},"by_repo_kind":{"official":{"samples":19,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"faa4a9c6af83199d","entry":"find_replicant","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_mem.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_mem.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"faa4a9c6af83199d"}},{"code_sha256_prefix":"6181848dd2a3e95a","entry":"generate_lattice_bfs","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_reid.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_reid.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6181848dd2a3e95a"}},{"code_sha256_prefix":"aa43b0b035541e98","entry":"get_entropy","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_attr.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_attr.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa43b0b035541e98"}},{"code_sha256_prefix":"70ae108e7faad492","entry":"prepare_data","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/synthetic_risk_model_utils.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/synthetic_risk_model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70ae108e7faad492"}},{"code_sha256_prefix":"e58e5ee7682cec31","entry":"prepare_data_vumc","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/synthetic_risk_model_utils.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/synthetic_risk_model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e58e5ee7682cec31"}},{"code_sha256_prefix":"00c49353fc807209","entry":"prob2onehot","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/dpgan.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/dpgan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00c49353fc807209"}},{"code_sha256_prefix":"53cd5ec558942f08","entry":"prob2onehot","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/emrwgan.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/emrwgan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53cd5ec558942f08"}},{"code_sha256_prefix":"1f74edbb78015691","entry":"reorder_data","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/synthetic_risk_model_utils.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/synthetic_risk_model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f74edbb78015691"}},{"code_sha256_prefix":"50ffdcf01c471e6d","entry":"cal_score","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_attr.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_attr.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50ffdcf01c471e6d"}},{"code_sha256_prefix":"426ef3a3832a4d1b","entry":"each_group","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_mem.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_mem.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"426ef3a3832a4d1b"}},{"code_sha256_prefix":"49019b0479f9788d","entry":"find_neighbour","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_attr.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_attr.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49019b0479f9788d"}},{"code_sha256_prefix":"30132d479215edc4","entry":"find_replicant_self","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_nnaa.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_nnaa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30132d479215edc4"}},{"code_sha256_prefix":"a9358df49bc7df73","entry":"gen","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/dpgan.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/dpgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9358df49bc7df73"}},{"code_sha256_prefix":"b2c6a7a16c604d07","entry":"gen","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/emrwgan.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/emrwgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2c6a7a16c604d07"}},{"code_sha256_prefix":"36bbb2edd2d1480b","entry":"generate_lattice_dfs","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_reid.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_reid.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"36bbb2edd2d1480b"}},{"code_sha256_prefix":"a9fddf4e63945063","entry":"main","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/ae.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/ae.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9fddf4e63945063"}},{"code_sha256_prefix":"32c073c452caea94","entry":"replace_dataset","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"privacy_evaluation/Synthetic_risk_model_reid.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/privacy_evaluation/Synthetic_risk_model_reid.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32c073c452caea94"}},{"code_sha256_prefix":"adeb9d566a32b488","entry":"train","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/dpgan.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/dpgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"adeb9d566a32b488"}},{"code_sha256_prefix":"501819ee8cd9f4a4","entry":"train","repo":"yy6linda/synthetic-ehr-benchmarking","repo_kind":"official","path":"data_generation/uw/emrwgan.py","file_url":"https://github.com/yy6linda/synthetic-ehr-benchmarking/blob/HEAD/data_generation/uw/emrwgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"501819ee8cd9f4a4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}