{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/synthetic-data-can-we-trust-statistical","title":"The Real Deal Behind the Artificial Appeal: Inferential Utility of Tabular Synthetic Data","arxiv_id":"2312.07837","date":"2023-12-13","proceeding":null,"authors":["Alexander Decruyenaere","Heidelinde Dehaene","Paloma Rabaey","Christiaan Polet","Johan Decruyenaere","Stijn Vansteelandt","Thomas Demeester"],"abstract":"Recent advances in generative models facilitate the creation of synthetic data to be made available for research in privacy-sensitive contexts. However, the analysis of synthetic data raises a unique set of methodological challenges. In this work, we highlight the importance of inferential utility and provide empirical evidence against naive inference from synthetic data, whereby synthetic data are treated as if they were actually observed. Before publishing synthetic data, it is essential to develop statistical inference tools for such data. By means of a simulation study, we show that the rate of false-positive findings (type 1 error) will be unacceptably high, even when the estimates are unbiased. Despite the use of a previously proposed correction factor, this problem persists for deep generative models, in part due to slower convergence of estimators and resulting underestimation of the true standard error. We further demonstrate our findings through a case study.","url_abs":"https://arxiv.org/abs/2312.07837v2","url_pdf":"https://arxiv.org/pdf/2312.07837v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"synthetic-data-can-we-trust-statistical","repo_url":"https://github.com/syndara-lab/inferential-utility","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.07837","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.07837"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/syndara-lab/inferential-utility","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"53c23384dd6a5fbb","entry":"create_dummies","repo":"syndara-lab/inferential-utility","repo_kind":"official","path":"sim_evaluate.py","file_url":"https://github.com/syndara-lab/inferential-utility/blob/HEAD/sim_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53c23384dd6a5fbb"}},{"code_sha256_prefix":"ad1f7331f8e54cc4","entry":"create_metadata","repo":"syndara-lab/inferential-utility","repo_kind":"official","path":"sim_evaluate.py","file_url":"https://github.com/syndara-lab/inferential-utility/blob/HEAD/sim_evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad1f7331f8e54cc4"}},{"code_sha256_prefix":"e07849e21c9ec2da","entry":"load_data","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e07849e21c9ec2da"}},{"code_sha256_prefix":"d50b876ed0b158af","entry":"sample_synthetic","repo":"syndara-lab/inferential-utility","repo_kind":"official","path":"sim_generate.py","file_url":"https://github.com/syndara-lab/inferential-utility/blob/HEAD/sim_generate.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d50b876ed0b158af"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}