{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-randomized-experiments-using","title":"Efficient Randomized Experiments Using Foundation Models","arxiv_id":"2502.04262","date":"2025-02-06","proceeding":null,"authors":["Piersilvio De Bartolomeis","Javier Abad","Guanbo Wang","Konstantin Donhauser","Raymond M. Duch","Fanny Yang","Issa J. Dahabreh"],"abstract":"Randomized experiments are the preferred approach for evaluating the effects of interventions, but they are costly and often yield estimates with substantial uncertainty. On the other hand, in silico experiments leveraging foundation models offer a cost-effective alternative that can potentially attain higher statistical precision. However, the benefits of in silico experiments come with a significant risk: statistical inferences are not valid if the models fail to accurately predict experimental responses to interventions. In this paper, we propose a novel approach that integrates the predictions from multiple foundation models with experimental data while preserving valid statistical inference. Our estimator is consistent and asymptotically normal, with asymptotic variance no larger than the standard estimator based on experimental data alone. Importantly, these statistical properties hold even when model predictions are arbitrarily biased. Empirical results across several randomized experiments show that our estimator offers substantial precision gains, equivalent to a reduction of up to 20% in the sample size needed to match the same precision as the standard estimator based on experimental data alone.","url_abs":"https://arxiv.org/abs/2502.04262v2","url_pdf":"https://arxiv.org/pdf/2502.04262v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-randomized-experiments-using","repo_url":"https://github.com/jaabmar/HAIPW","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"efficient-randomized-experiments-using","repo_url":"https://github.com/jaabmar/RAMEN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.04262","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.04262"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jaabmar/RAMEN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jaabmar/HAIPW","reach":null}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e77ae3d5122717fd","entry":"GumbelGate","repo":"jaabmar/RAMEN","repo_kind":"listed","path":"RAMEN/models/insta_ramen.py","file_url":"https://github.com/jaabmar/RAMEN/blob/HEAD/RAMEN/models/insta_ramen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e77ae3d5122717fd"}},{"code_sha256_prefix":"7af252e6572255ff","entry":"HAIPWEstimator","repo":"jaabmar/HAIPW","repo_kind":"official","path":"HAIPW/estimators.py","file_url":"https://github.com/jaabmar/HAIPW/blob/HEAD/HAIPW/estimators.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7af252e6572255ff"}},{"code_sha256_prefix":"c4c2c5a20b2de9c7","entry":"NeuralNet","repo":"jaabmar/RAMEN","repo_kind":"listed","path":"RAMEN/models/insta_ramen.py","file_url":"https://github.com/jaabmar/RAMEN/blob/HEAD/RAMEN/models/insta_ramen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4c2c5a20b2de9c7"}},{"code_sha256_prefix":"fcb065229b17f2b0","entry":"Ramen","repo":"jaabmar/RAMEN","repo_kind":"listed","path":"RAMEN/models/insta_ramen.py","file_url":"https://github.com/jaabmar/RAMEN/blob/HEAD/RAMEN/models/insta_ramen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fcb065229b17f2b0"}},{"code_sha256_prefix":"9ab00ef658b79fb0","entry":"BaseEstimator","repo":"jaabmar/HAIPW","repo_kind":"official","path":"HAIPW/estimators.py","file_url":"https://github.com/jaabmar/HAIPW/blob/HEAD/HAIPW/estimators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ab00ef658b79fb0"}},{"code_sha256_prefix":"c3cb779dbb0e34a2","entry":"InstaRamen","repo":"jaabmar/RAMEN","repo_kind":"listed","path":"RAMEN/models/insta_ramen.py","file_url":"https://github.com/jaabmar/RAMEN/blob/HEAD/RAMEN/models/insta_ramen.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3cb779dbb0e34a2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}