{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pareto-invariant-risk-minimization","title":"Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalization","arxiv_id":"2206.07766","date":"2022-06-15","proceeding":null,"authors":["Yongqiang Chen","Kaiwen Zhou","Yatao Bian","Binghui Xie","Bingzhe Wu","Yonggang Zhang","Kaili Ma","Han Yang","Peilin Zhao","Bo Han","James Cheng"],"abstract":"Recently, there has been a growing surge of interest in enabling machine learning systems to generalize well to Out-of-Distribution (OOD) data. Most efforts are devoted to advancing optimization objectives that regularize models to capture the underlying invariance; however, there often are compromises in the optimization process of these OOD objectives: i) Many OOD objectives have to be relaxed as penalty terms of Empirical Risk Minimization (ERM) for the ease of optimization, while the relaxed forms can weaken the robustness of the original objective; ii) The penalty terms also require careful tuning of the penalty weights due to the intrinsic conflicts between ERM and OOD objectives. Consequently, these compromises could easily lead to suboptimal performance of either the ERM or OOD objective. To address these issues, we introduce a multi-objective optimization (MOO) perspective to understand the OOD optimization process, and propose a new optimization scheme called PAreto Invariant Risk Minimization (PAIR). PAIR improves the robustness of OOD objectives by cooperatively optimizing with other OOD objectives, thereby bridging the gaps caused by the relaxations. Then PAIR approaches a Pareto optimal solution that trades off the ERM and OOD objectives properly. Extensive experiments on challenging benchmarks, WILDS, show that PAIR alleviates the compromises and yields top OOD performances.","url_abs":"https://arxiv.org/abs/2206.07766v2","url_pdf":"https://arxiv.org/pdf/2206.07766v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pareto-invariant-risk-minimization","repo_url":"https://github.com/lfhase/pair","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"pareto-invariant-risk-minimization","repo_url":"https://github.com/lfhase/ciga","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"pareto-invariant-risk-minimization","repo_url":"https://github.com/qkrdmsghk/GOODHSE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"pareto-invariant-risk-minimization","repo_url":"https://github.com/MindSpore-scientific-2/code-1/tree/main/PACMOO","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2206.07766","atlas_url":"https://app.syntology.ai/?focus=2206.07766","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.07766"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific-2/code-1/tree/main/PACMOO","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qkrdmsghk/GOODHSE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lfhase/pair","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lfhase/ciga","reach":null}],"summary":{"ran":2,"ran_honours":1,"ran_draft_wrong":2,"ran_violates":1,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1},"listed":{"samples":1,"ran":0,"repositories":1},"community":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"d16a88da846b7682","entry":"EPO","repo":"lfhase/pair","repo_kind":"official","path":"PAIR/pair.py","file_url":"https://github.com/lfhase/pair/blob/HEAD/PAIR/pair.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d16a88da846b7682"}},{"code_sha256_prefix":"8c73c6ae98d61657","entry":"SEPO","repo":"lfhase/pair","repo_kind":"official","path":"PAIR/pair.py","file_url":"https://github.com/lfhase/pair/blob/HEAD/PAIR/pair.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c73c6ae98d61657"}},{"code_sha256_prefix":"28077f9c052f7aed","entry":"getNumParams","repo":"lfhase/pair","repo_kind":"official","path":"ColoredMNIST/pair.py","file_url":"https://github.com/lfhase/pair/blob/HEAD/ColoredMNIST/pair.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"28077f9c052f7aed"}},{"code_sha256_prefix":"fc7bb9ff36d7b1f1","entry":"get_irm_loss","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fc7bb9ff36d7b1f1"}},{"code_sha256_prefix":"cb517763d1083872","entry":"get_kl_div","repo":"lfhase/pair","repo_kind":"official","path":"ColoredMNIST/pair.py","file_url":"https://github.com/lfhase/pair/blob/HEAD/ColoredMNIST/pair.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb517763d1083872"}},{"code_sha256_prefix":"32a5b544ec6bebb1","entry":"pair_selection","repo":"lfhase/pair","repo_kind":"official","path":"ColoredMNIST/pair.py","file_url":"https://github.com/lfhase/pair/blob/HEAD/ColoredMNIST/pair.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32a5b544ec6bebb1"}},{"code_sha256_prefix":"d0aa27294b3dd8c9","entry":"MinNormSolver","repo":"MindSpore-scientific-2/code-1","repo_kind":"listed","path":"PACMOO/min_norm_solvers_numpy.py","file_url":"https://github.com/MindSpore-scientific-2/code-1/blob/HEAD/PACMOO/min_norm_solvers_numpy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d0aa27294b3dd8c9"}},{"code_sha256_prefix":"eca51e17518810ff","entry":"PAIR","repo":"lfhase/pair","repo_kind":"official","path":"PAIR/pair.py","file_url":"https://github.com/lfhase/pair/blob/HEAD/PAIR/pair.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eca51e17518810ff"}},{"code_sha256_prefix":"93721419ca48ae8f","entry":"init_input_canvas_wrapper","repo":"Picsart-AI-Research/PAIR-Diffusion","repo_kind":"community","path":"gradio_demo.py","file_url":"https://github.com/Picsart-AI-Research/PAIR-Diffusion/blob/HEAD/gradio_demo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93721419ca48ae8f"}},{"code_sha256_prefix":"a6787f2913c02b9a","entry":"init_ref_canvas_wrapper","repo":"Picsart-AI-Research/PAIR-Diffusion","repo_kind":"community","path":"gradio_demo.py","file_url":"https://github.com/Picsart-AI-Research/PAIR-Diffusion/blob/HEAD/gradio_demo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6787f2913c02b9a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}