{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diverse-weight-averaging-for-out-of","title":"Diverse Weight Averaging for Out-of-Distribution Generalization","arxiv_id":"2205.09739","date":"2022-05-19","proceeding":null,"authors":["Alexandre Ramé","Matthieu Kirchmeyer","Thibaud Rahier","Alain Rakotomamonjy","Patrick Gallinari","Matthieu Cord"],"abstract":"Standard neural networks struggle to generalize under distribution shifts in computer vision. Fortunately, combining multiple networks can consistently improve out-of-distribution generalization. In particular, weight averaging (WA) strategies were shown to perform best on the competitive DomainBed benchmark; they directly average the weights of multiple networks despite their nonlinearities. In this paper, we propose Diverse Weight Averaging (DiWA), a new WA strategy whose main motivation is to increase the functional diversity across averaged models. To this end, DiWA averages weights obtained from several independent training runs: indeed, models obtained from different runs are more diverse than those collected along a single run thanks to differences in hyperparameters and training procedures. We motivate the need for diversity by a new bias-variance-covariance-locality decomposition of the expected error, exploiting similarities between WA and standard functional ensembling. Moreover, this decomposition highlights that WA succeeds when the variance term dominates, which we show occurs when the marginal distribution changes at test time. Experimentally, DiWA consistently improves the state of the art on DomainBed without inference overhead.","url_abs":"https://arxiv.org/abs/2205.09739v2","url_pdf":"https://arxiv.org/pdf/2205.09739v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diverse-weight-averaging-for-out-of","repo_url":"https://github.com/alexrame/diwa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"diverse-weight-averaging-for-out-of","repo_url":"https://github.com/facebookresearch/ModelRatatouille","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2205.09739","atlas_url":"https://app.syntology.ai/?focus=2205.09739","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.09739"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alexrame/diwa","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/ModelRatatouille","reach":null}],"summary":{"ran_draft_wrong":3,"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":12,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ce7990d7ad5821ff","entry":"Classifier","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/networks.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/networks.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce7990d7ad5821ff"}},{"code_sha256_prefix":"00e569acd6b45ef0","entry":"conv3x3","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/lib/wide_resnet.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/lib/wide_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00e569acd6b45ef0"}},{"code_sha256_prefix":"837d97171609bbbf","entry":"get_score","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/lib/misc.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/lib/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"837d97171609bbbf"}},{"code_sha256_prefix":"53fac8d8d949e72b","entry":"get_test_records","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/model_selection.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/model_selection.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"53fac8d8d949e72b"}},{"code_sha256_prefix":"71a3a61ceed99bf3","entry":"hashable","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/lib/query.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/lib/query.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"71a3a61ceed99bf3"}},{"code_sha256_prefix":"1fdaff04251a0d51","entry":"l2_between_dicts","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/lib/misc.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/lib/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1fdaff04251a0d51"}},{"code_sha256_prefix":"2f8af5779e1edcb6","entry":"make_selector_fn","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/lib/query.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/lib/query.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f8af5779e1edcb6"}},{"code_sha256_prefix":"5f576ed342c48a0a","entry":"make_weights_for_balanced_classes","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/lib/misc.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/lib/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5f576ed342c48a0a"}},{"code_sha256_prefix":"196cab71d7129d62","entry":"remove_batch_norm_from_resnet","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/networks.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/networks.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"196cab71d7129d62"}},{"code_sha256_prefix":"b0bc80b1655a6802","entry":"get_algorithm_class","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/algorithms.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/algorithms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0bc80b1655a6802"}},{"code_sha256_prefix":"d0ea85d74c20dea9","entry":"get_dataset_class","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/datasets.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d0ea85d74c20dea9"}},{"code_sha256_prefix":"73f32252eedba6a4","entry":"num_environments","repo":"alexrame/diwa","repo_kind":"official","path":"domainbed/datasets.py","file_url":"https://github.com/alexrame/diwa/blob/HEAD/domainbed/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73f32252eedba6a4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}