{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-investigation-of-why-overparameterization","title":"An Investigation of Why Overparameterization Exacerbates Spurious Correlations","arxiv_id":"2005.04345","date":"2020-05-09","proceeding":null,"authors":["Shiori Sagawa","aditi raghunathan","Pang Wei Koh","Percy Liang"],"abstract":"We study why overparameterization -- increasing model size well beyond the point of zero training error -- can hurt test error on minority groups despite improving average test error when there are spurious correlations in the data. Through simulations and experiments on two image datasets, we identify two key properties of the training data that drive this behavior: the proportions of majority versus minority groups, and the signal-to-noise ratio of the spurious correlations. We then analyze a linear setting and theoretically show how the inductive bias of models towards \"memorizing\" fewer examples can cause overparameterization to hurt. Our analysis leads to a counterintuitive approach of subsampling the majority group, which empirically achieves low minority error in the overparameterized regime, even though the standard approach of upweighting the minority fails. Overall, our results suggest a tension between using overparameterized models versus using all the training data for achieving low worst-group error.","url_abs":"https://arxiv.org/abs/2005.04345v3","url_pdf":"https://arxiv.org/pdf/2005.04345v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-investigation-of-why-overparameterization","repo_url":"https://github.com/ssagawa/overparam_spur_corr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"an-investigation-of-why-overparameterization","repo_url":"https://worksheets.codalab.org/worksheets/0x1db77e603a8d48c8abebd67fce39cf8b","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"an-investigation-of-why-overparameterization","repo_url":"https://github.com/qiaoruiyt/noiserobustdg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.04345","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.04345"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ssagawa/overparam_spur_corr","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qiaoruiyt/noiserobustdg","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://worksheets.codalab.org/worksheets/0x1db77e603a8d48c8abebd67fce39cf8b","reach":null}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"fbd312e0a0991c68","entry":"load_waterbirds_data","repo":"ssagawa/overparam_spur_corr","repo_kind":"official","path":"run_waterbirds_random_features.py","file_url":"https://github.com/ssagawa/overparam_spur_corr/blob/HEAD/run_waterbirds_random_features.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbd312e0a0991c68"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}