{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/predictors-from-causal-features-do-not","title":"Do causal predictors generalize better to new domains?","arxiv_id":"2402.09891","date":"2024-02-15","proceeding":null,"authors":["Vivian Y. Nastl","Moritz Hardt"],"abstract":"We study how well machine learning models trained on causal features generalize across domains. We consider 16 prediction tasks on tabular datasets covering applications in health, employment, education, social benefits, and politics. Each dataset comes with multiple domains, allowing us to test how well a model trained in one domain performs in another. For each prediction task, we select features that have a causal influence on the target of prediction. Our goal is to test the hypothesis that models trained on causal features generalize better across domains. Without exception, we find that predictors using all available features, regardless of causality, have better in-domain and out-of-domain accuracy than predictors using causal features. Moreover, even the absolute drop in accuracy from one domain to the other is no better for causal predictors than for models that use all features. In addition, we show that recent causal machine learning methods for domain generalization do not perform better in our evaluation than standard predictors trained on the set of causal features. Likewise, causal discovery algorithms either fail to run or select causal variables that perform no better than our selection. Extensive robustness checks confirm that our findings are stable under variable misclassification.","url_abs":"https://arxiv.org/abs/2402.09891v2","url_pdf":"https://arxiv.org/pdf/2402.09891v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"predictors-from-causal-features-do-not","repo_url":"https://github.com/socialfoundations/causal-features","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"causal-discovery","task_name":"Causal Discovery"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.09891","atlas_url":"https://app.syntology.ai/?focus=2402.09891","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.09891"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/socialfoundations/causal-features","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/py-why/dodiscover","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":12},"by_repo_kind":{"official":{"samples":7,"ran":7,"repositories":1},"found_in_text":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"4b205fa5034b6cac","entry":"IS_TASK_DG","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/launch_one_experiment.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/launch_one_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4b205fa5034b6cac"}},{"code_sha256_prefix":"c32a1c87fac066b9","entry":"balanced_accuracy_score","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/metrics.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c32a1c87fac066b9"}},{"code_sha256_prefix":"332453e98a97f129","entry":"binary_stat_scores_format","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/metrics.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"332453e98a97f129"}},{"code_sha256_prefix":"7ef2910fa357bf57","entry":"f_divergence_score","repo":"py-why/dodiscover","repo_kind":"found_in_text","path":"dodiscover/ci/kernel_utils.py","file_url":"https://github.com/py-why/dodiscover/blob/HEAD/dodiscover/ci/kernel_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ef2910fa357bf57"}},{"code_sha256_prefix":"adc805a8edd55f21","entry":"generate_knn_in_subspace","repo":"py-why/dodiscover","repo_kind":"found_in_text","path":"dodiscover/ci/monte_carlo.py","file_url":"https://github.com/py-why/dodiscover/blob/HEAD/dodiscover/ci/monte_carlo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"adc805a8edd55f21"}},{"code_sha256_prefix":"48c885404b9c9689","entry":"get_dic_experiments_value","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/plot_experiment.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/plot_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"48c885404b9c9689"}},{"code_sha256_prefix":"c5b10bd8cd459cf0","entry":"get_dic_experiments_value","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/plot_experiment_arguablycausal_robust.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/plot_experiment_arguablycausal_robust.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c5b10bd8cd459cf0"}},{"code_sha256_prefix":"eda5e96ba9a34cd0","entry":"get_dic_experiments_value","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/plot_experiment_balanced.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/plot_experiment_balanced.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"eda5e96ba9a34cd0"}},{"code_sha256_prefix":"58bcc6c74ba0ec3f","entry":"get_dic_experiments_value_anticausal","repo":"socialfoundations/causal-features","repo_kind":"official","path":"experiments_causal/plot_experiment_anticausal.py","file_url":"https://github.com/socialfoundations/causal-features/blob/HEAD/experiments_causal/plot_experiment_anticausal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"58bcc6c74ba0ec3f"}},{"code_sha256_prefix":"a371b4ee094ac18d","entry":"kl_divergence_score","repo":"py-why/dodiscover","repo_kind":"found_in_text","path":"dodiscover/ci/kernel_utils.py","file_url":"https://github.com/py-why/dodiscover/blob/HEAD/dodiscover/ci/kernel_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a371b4ee094ac18d"}},{"code_sha256_prefix":"3755a6e00fae05ea","entry":"restricted_nbr_permutation","repo":"py-why/dodiscover","repo_kind":"found_in_text","path":"dodiscover/ci/monte_carlo.py","file_url":"https://github.com/py-why/dodiscover/blob/HEAD/dodiscover/ci/monte_carlo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3755a6e00fae05ea"}},{"code_sha256_prefix":"e3bf1f934a3c61cd","entry":"von_neumann_divergence","repo":"py-why/dodiscover","repo_kind":"found_in_text","path":"dodiscover/ci/kernel_utils.py","file_url":"https://github.com/py-why/dodiscover/blob/HEAD/dodiscover/ci/kernel_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3bf1f934a3c61cd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}