{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/empirical-or-invariant-risk-minimization-a-1","title":"Empirical or Invariant Risk Minimization? A Sample Complexity Perspective","arxiv_id":"2010.16412","date":"2020-10-30","proceeding":"ICLR 2021 1","authors":["Kartik Ahuja","Jun Wang","Amit Dhurandhar","Karthikeyan Shanmugam","Kush R. Varshney"],"abstract":"Recently, invariant risk minimization (IRM) was proposed as a promising solution to address out-of-distribution (OOD) generalization. However, it is unclear when IRM should be preferred over the widely-employed empirical risk minimization (ERM) framework. In this work, we analyze both these frameworks from the perspective of sample complexity, thus taking a firm step towards answering this important question. We find that depending on the type of data generation mechanism, the two approaches might have very different finite sample and asymptotic behavior. For example, in the covariate shift setting we see that the two approaches not only arrive at the same asymptotic solution, but also have similar finite sample behavior with no clear winner. For other distribution shifts such as those involving confounders or anti-causal variables, however, the two approaches arrive at different asymptotic solutions where IRM is guaranteed to be close to the desired OOD solutions in the finite sample regime, while ERM is biased even asymptotically. We further investigate how different factors -- the number of environments, complexity of the model, and IRM penalty weight -- impact the sample complexity of IRM in relation to its distance from the OOD solutions","url_abs":"https://arxiv.org/abs/2010.16412v2","url_pdf":"https://arxiv.org/pdf/2010.16412v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"empirical-or-invariant-risk-minimization-a-1","repo_url":"https://github.com/IBM/IRM-games","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"empirical-or-invariant-risk-minimization-a-1","repo_url":"https://github.com/IBM/OoD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"empirical-or-invariant-risk-minimization-a-1","repo_url":"https://github.com/IBM/OoD/tree/master/ERM-IRM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2010.16412","atlas_url":"https://app.syntology.ai/?focus=2010.16412","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.16412"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/OoD/tree/master/ERM-IRM","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/IRM-games","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/OoD","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"unverified":12},"by_repo_kind":{"official":{"samples":14,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"db4d05619d1e83d4","entry":"pretty","repo":"IBM/IRM-games","repo_kind":"official","path":"ERM-IRM/models_crossval_Sep8.py","file_url":"https://github.com/IBM/IRM-games/blob/HEAD/ERM-IRM/models_crossval_Sep8.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"db4d05619d1e83d4"}},{"code_sha256_prefix":"1463989a26f3aa4f","entry":"pretty","repo":"IBM/IRM-games","repo_kind":"official","path":"LRG_games/ERM_per_env/IRM_exhaustive_argparse_hper_wo_disc_Sep8.py","file_url":"https://github.com/IBM/IRM-games/blob/HEAD/LRG_games/ERM_per_env/IRM_exhaustive_argparse_hper_wo_disc_Sep8.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1463989a26f3aa4f"}},{"code_sha256_prefix":"a2f985068c422b43","entry":"ERM_Sblock","repo":"IBM/IRM-games","repo_kind":"official","path":"IRM_ITE/models.py","file_url":"https://github.com/IBM/IRM-games/blob/HEAD/IRM_ITE/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2f985068c422b43"}},{"code_sha256_prefix":"a7f23fec9f83e15e","entry":"ERM_Tblock","repo":"IBM/IRM-games","repo_kind":"official","path":"IRM_ITE/models.py","file_url":"https://github.com/IBM/IRM-games/blob/HEAD/IRM_ITE/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a7f23fec9f83e15e"}},{"code_sha256_prefix":"4dbfd6b03c7a8067","entry":"IRM_Sblock","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/irm_block.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/irm_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4dbfd6b03c7a8067"}},{"code_sha256_prefix":"3ef447ce6417f2a3","entry":"PEHE","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/metrics.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3ef447ce6417f2a3"}},{"code_sha256_prefix":"91d3cbc506dacd10","entry":"ate_error","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/metrics.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"91d3cbc506dacd10"}},{"code_sha256_prefix":"cabc72b51279fac0","entry":"convert_regn_np_format","repo":"IBM/IRM-games","repo_kind":"official","path":"LRG_games/irm_games_regression.py","file_url":"https://github.com/IBM/IRM-games/blob/HEAD/LRG_games/irm_games_regression.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cabc72b51279fac0"}},{"code_sha256_prefix":"c84217ccfbad36a6","entry":"envs_irm_S","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/irm_block.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/irm_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c84217ccfbad36a6"}},{"code_sha256_prefix":"57e301bf69860e40","entry":"envs_irm_T","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/irm_block.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/irm_block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"57e301bf69860e40"}},{"code_sha256_prefix":"b73c6efcdaeee963","entry":"errors","repo":"IBM/IRM-games","repo_kind":"official","path":"LRG_games/ERM_per_env/IRM_exhaustive_argparse_hper_wo_disc_Sep8.py","file_url":"https://github.com/IBM/IRM-games/blob/HEAD/LRG_games/ERM_per_env/IRM_exhaustive_argparse_hper_wo_disc_Sep8.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b73c6efcdaeee963"}},{"code_sha256_prefix":"b46d230fde0188cd","entry":"generate_T","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/generate_synthetic_data.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/generate_synthetic_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b46d230fde0188cd"}},{"code_sha256_prefix":"e39c919dc88468e7","entry":"generate_mixture_indicator","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/generate_synthetic_data.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/generate_synthetic_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e39c919dc88468e7"}},{"code_sha256_prefix":"fc6dfd875f58ef25","entry":"generate_synthetic_data","repo":"IBM/OoD","repo_kind":"official","path":"IRM_ITE/generate_synthetic_data.py","file_url":"https://github.com/IBM/OoD/blob/HEAD/IRM_ITE/generate_synthetic_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc6dfd875f58ef25"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}