{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/falsification-before-extrapolation-in-causal","title":"Falsification before Extrapolation in Causal Effect Estimation","arxiv_id":"2209.13708","date":"2022-09-27","proceeding":null,"authors":["Zeshan Hussain","Michael Oberst","Ming-Chieh Shih","David Sontag"],"abstract":"Randomized Controlled Trials (RCTs) represent a gold standard when developing policy guidelines. However, RCTs are often narrow, and lack data on broader populations of interest. Causal effects in these populations are often estimated using observational datasets, which may suffer from unobserved confounding and selection bias. Given a set of observational estimates (e.g. from multiple studies), we propose a meta-algorithm that attempts to reject observational estimates that are biased. We do so using validation effects, causal effects that can be inferred from both RCT and observational data. After rejecting estimators that do not pass this test, we generate conservative confidence intervals on the extrapolated causal effects for subgroups not observed in the RCT. Under the assumption that at least one observational estimator is asymptotically normal and consistent for both the validation and extrapolated effects, we provide guarantees on the coverage probability of the intervals output by our algorithm. To facilitate hypothesis testing in settings where causal effect transportation across datasets is necessary, we give conditions under which a doubly-robust estimator of group average treatment effects is asymptotically normal, even when flexible machine learning methods are used for estimation of nuisance parameters. We illustrate the properties of our approach on semi-synthetic and real world datasets, and show that it compares favorably to standard meta-analysis techniques.","url_abs":"https://arxiv.org/abs/2209.13708v3","url_pdf":"https://arxiv.org/pdf/2209.13708v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"falsification-before-extrapolation-in-causal","repo_url":"https://github.com/clinicalml/rct-obs-extrapolation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"selection-bias","task_name":"Selection bias"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.13708","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.13708"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/clinicalml/rct-obs-extrapolation","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2eed732ab6666408","entry":"get_strata_Ns","repo":"clinicalml/rct-obs-extrapolation","repo_kind":"official","path":"experiments/whi_experiment.py","file_url":"https://github.com/clinicalml/rct-obs-extrapolation/blob/HEAD/experiments/whi_experiment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2eed732ab6666408"}},{"code_sha256_prefix":"ffdad971e2a6d26a","entry":"get_strata_info","repo":"clinicalml/rct-obs-extrapolation","repo_kind":"official","path":"experiments/whi_experiment.py","file_url":"https://github.com/clinicalml/rct-obs-extrapolation/blob/HEAD/experiments/whi_experiment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffdad971e2a6d26a"}},{"code_sha256_prefix":"ce3a242febe58da7","entry":"read_yaml","repo":"clinicalml/rct-obs-extrapolation","repo_kind":"official","path":"experiments/util.py","file_url":"https://github.com/clinicalml/rct-obs-extrapolation/blob/HEAD/experiments/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce3a242febe58da7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}