{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pulling-up-by-the-causal-bootstraps-causal","title":"Pulling Up by the Causal Bootstraps: Causal Data Augmentation for Pre-training Debiasing","arxiv_id":"2108.12510","date":"2021-08-27","proceeding":null,"authors":["Sindhu C. M. Gowda","Shalmali Joshi","Haoran Zhang","Marzyeh Ghassemi"],"abstract":"Machine learning models achieve state-of-the-art performance on many supervised learning tasks. However, prior evidence suggests that these models may learn to rely on shortcut biases or spurious correlations (intuitively, correlations that do not hold in the test as they hold in train) for good predictive performance. Such models cannot be trusted in deployment environments to provide accurate predictions. While viewing the problem from a causal lens is known to be useful, the seamless integration of causation techniques into machine learning pipelines remains cumbersome and expensive. In this work, we study and extend a causal pre-training debiasing technique called causal bootstrapping (CB) under five practical confounded-data generation-acquisition scenarios (with known and unknown confounding). Under these settings, we systematically investigate the effect of confounding bias on deep learning model performance, demonstrating their propensity to rely on shortcut biases when these biases are not properly accounted for. We demonstrate that such a causal pre-training technique can significantly outperform existing base practices to mitigate confounding bias on real-world domain generalization benchmarking tasks. This systematic investigation underlines the importance of accounting for the underlying data-generating mechanisms and fortifying data-preprocessing pipelines with a causal framework to develop methods robust to confounding biases.","url_abs":"https://arxiv.org/abs/2108.12510v1","url_pdf":"https://arxiv.org/pdf/2108.12510v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pulling-up-by-the-causal-bootstraps-causal","repo_url":"https://github.com/MLforHealth/CausalDA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2108.12510","atlas_url":"https://app.syntology.ai/?focus=2108.12510","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.12510"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MLforHealth/CausalDA","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":2,"unverified":13},"by_repo_kind":{"official":{"samples":17,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0946df67b41ca7e6","entry":"block_until_running","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"launchers.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/launchers.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0946df67b41ca7e6"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"model/resnet_multispectral.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/model/resnet_multispectral.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"model/resnet_multispectral.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/model/resnet_multispectral.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"501459823473afca","entry":"get_num_jobs","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"launchers.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/launchers.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"501459823473afca"}},{"code_sha256_prefix":"ef58194dbb437756","entry":"combinations","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"experiments.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef58194dbb437756"}},{"code_sha256_prefix":"820b345e20bfded6","entry":"cost_fn","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"src/prediction.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/src/prediction.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"820b345e20bfded6"}},{"code_sha256_prefix":"daf19afe2e91d4a5","entry":"cost_fn","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"src/training.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/src/training.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"daf19afe2e91d4a5"}},{"code_sha256_prefix":"1333b4e802eba4f0","entry":"get_hparams","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"experiments.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1333b4e802eba4f0"}},{"code_sha256_prefix":"352f597b6a390f3a","entry":"get_script_name","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"experiments.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"352f597b6a390f3a"}},{"code_sha256_prefix":"d2d5ec732e91235f","entry":"prediction_analysis","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"src/prediction.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/src/prediction.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2d5ec732e91235f"}},{"code_sha256_prefix":"ded081dd38c7c24a","entry":"random_subset_seq","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"train_env.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/train_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ded081dd38c7c24a"}},{"code_sha256_prefix":"a3ceab17d25e15b2","entry":"simple_models","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"model/simple_models.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/model/simple_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3ceab17d25e15b2"}},{"code_sha256_prefix":"5c0450ec429262a9","entry":"split_dataset","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"train_env.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/train_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c0450ec429262a9"}},{"code_sha256_prefix":"9cbb9737a804d9ab","entry":"train","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"src/training.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/src/training.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9cbb9737a804d9ab"}},{"code_sha256_prefix":"c146534ad54e89c3","entry":"train_step","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"src/training.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/src/training.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c146534ad54e89c3"}},{"code_sha256_prefix":"85d5efbf0175a369","entry":"validation","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"src/validation.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/src/validation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"85d5efbf0175a369"}},{"code_sha256_prefix":"cac3223f3d4a4a25","entry":"wilds_get_merged_dataset","repo":"MLforHealth/CausalDA","repo_kind":"official","path":"train_env.py","file_url":"https://github.com/MLforHealth/CausalDA/blob/HEAD/train_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cac3223f3d4a4a25"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}