{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adjustment-for-confounding-using-pre-trained","title":"Adjustment for Confounding using Pre-Trained Representations","arxiv_id":"2506.14329","date":"2025-06-17","proceeding":null,"authors":["Rickmer Schulte","David Rügamer","Thomas Nagler"],"abstract":"There is growing interest in extending average treatment effect (ATE) estimation to incorporate non-tabular data, such as images and text, which may act as sources of confounding. Neglecting these effects risks biased results and flawed scientific conclusions. However, incorporating non-tabular data necessitates sophisticated feature extractors, often in combination with ideas of transfer learning. In this work, we investigate how latent features from pre-trained neural networks can be leveraged to adjust for sources of confounding. We formalize conditions under which these latent features enable valid adjustment and statistical inference in ATE estimation, demonstrating results along the example of double machine learning. We discuss critical challenges inherent to latent feature learning and downstream parameter estimation arising from the high dimensionality and non-identifiability of representations. Common structural assumptions for obtaining fast convergence rates with additive or sparse linear models are shown to be unrealistic for latent features. We argue, however, that neural networks are largely insensitive to these issues. In particular, we show that neural networks can achieve fast convergence rates by adapting to intrinsic notions of sparsity and dimension of the learning problem.","url_abs":"https://arxiv.org/abs/2506.14329v1","url_pdf":"https://arxiv.org/pdf/2506.14329v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adjustment-for-confounding-using-pre-trained","repo_url":"https://github.com/rickmer-schulte/pretrained-causal-adjust","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"parameter-estimation","task_name":"parameter estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.14329","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.14329"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/rickmer-schulte/Pretrained-Causal-Adjust","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rickmer-schulte/pretrained-causal-adjust","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"edbf00ba07cb1a88","entry":"collate_batch","repo":"rickmer-schulte/Pretrained-Causal-Adjust","repo_kind":"official","path":"src/data_setup/create_data_imdb.py","file_url":"https://github.com/rickmer-schulte/Pretrained-Causal-Adjust/blob/HEAD/src/data_setup/create_data_imdb.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"edbf00ba07cb1a88"}},{"code_sha256_prefix":"60c48af99a755db8","entry":"collect_images","repo":"rickmer-schulte/Pretrained-Causal-Adjust","repo_kind":"official","path":"src/data_setup/remove_duplicates.py","file_url":"https://github.com/rickmer-schulte/Pretrained-Causal-Adjust/blob/HEAD/src/data_setup/remove_duplicates.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60c48af99a755db8"}},{"code_sha256_prefix":"f1bc9a163ad18f18","entry":"create_block_diagonal_rotation_matrix","repo":"rickmer-schulte/Pretrained-Causal-Adjust","repo_kind":"official","path":"src/helpers/feature_rotation.py","file_url":"https://github.com/rickmer-schulte/Pretrained-Causal-Adjust/blob/HEAD/src/helpers/feature_rotation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1bc9a163ad18f18"}},{"code_sha256_prefix":"00025b9d92164af5","entry":"fit_autoencoder","repo":"rickmer-schulte/Pretrained-Causal-Adjust","repo_kind":"official","path":"src/helpers/ae.py","file_url":"https://github.com/rickmer-schulte/Pretrained-Causal-Adjust/blob/HEAD/src/helpers/ae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00025b9d92164af5"}},{"code_sha256_prefix":"3a5bed001d14b231","entry":"rotate_latent_features","repo":"rickmer-schulte/Pretrained-Causal-Adjust","repo_kind":"official","path":"src/helpers/feature_rotation.py","file_url":"https://github.com/rickmer-schulte/Pretrained-Causal-Adjust/blob/HEAD/src/helpers/feature_rotation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a5bed001d14b231"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}