{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/population-parameter-averaging-papa","title":"PopulAtion Parameter Averaging (PAPA)","arxiv_id":"2304.03094","date":"2023-04-06","proceeding":null,"authors":["Alexia Jolicoeur-Martineau","Emy Gervais","Kilian Fatras","Yan Zhang","Simon Lacoste-Julien"],"abstract":"Ensemble methods combine the predictions of multiple models to improve performance, but they require significantly higher computation costs at inference time. To avoid these costs, multiple neural networks can be combined into one by averaging their weights. However, this usually performs significantly worse than ensembling. Weight averaging is only beneficial when different enough to benefit from combining them, but similar enough to average well. Based on this idea, we propose PopulAtion Parameter Averaging (PAPA): a method that combines the generality of ensembling with the efficiency of weight averaging. PAPA leverages a population of diverse models (trained on different data orders, augmentations, and regularizations) while slowly pushing the weights of the networks toward the population average of the weights. We also propose PAPA variants (PAPA-all, and PAPA-2) that average weights rarely rather than continuously; all methods increase generalization, but PAPA tends to perform best. PAPA reduces the performance gap between averaging and ensembling, increasing the average accuracy of a population of models by up to 0.8% on CIFAR-10, 1.9% on CIFAR-100, and 1.6% on ImageNet when compared to training independent (non-averaged) models.","url_abs":"https://arxiv.org/abs/2304.03094v4","url_pdf":"https://arxiv.org/pdf/2304.03094v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"population-parameter-averaging-papa","repo_url":"https://github.com/samsungsailmontreal/papa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.03094","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.03094"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/samsungsailmontreal/papa","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":11},"by_repo_kind":{"official":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d7c3eb24eb759696","entry":"ResNet18","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7c3eb24eb759696"}},{"code_sha256_prefix":"ca893f9498143980","entry":"conv3x3","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca893f9498143980"}},{"code_sha256_prefix":"9cd45e4b4fb9ad28","entry":"convert_model","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"sync_batchnorm/batchnorm.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/sync_batchnorm/batchnorm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9cd45e4b4fb9ad28"}},{"code_sha256_prefix":"b2174c34ed70a43a","entry":"get_blocks","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2174c34ed70a43a"}},{"code_sha256_prefix":"af037d161636f160","entry":"get_layer_perm","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"recombine/corr_matching.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/recombine/corr_matching.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af037d161636f160"}},{"code_sha256_prefix":"f6cae5f4d5272117","entry":"get_layer_perm1","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"recombine/corr_matching.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/recombine/corr_matching.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6cae5f4d5272117"}},{"code_sha256_prefix":"6cf54613a40deb52","entry":"make_repaired_net","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"recombine/repair.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/recombine/repair.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cf54613a40deb52"}},{"code_sha256_prefix":"1601cfedc4402bd6","entry":"make_tracked_net","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"recombine/repair.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/recombine/repair.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1601cfedc4402bd6"}},{"code_sha256_prefix":"d22d36bb8ee5d00b","entry":"mix_weights_direct","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"recombine/permutations.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/recombine/permutations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d22d36bb8ee5d00b"}},{"code_sha256_prefix":"5ec019fd36dd68e8","entry":"resnet20","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"models/resnet20.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/models/resnet20.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ec019fd36dd68e8"}},{"code_sha256_prefix":"527355ff9a678a4c","entry":"run_corr_matrix","repo":"samsungsailmontreal/papa","repo_kind":"official","path":"recombine/corr_matching.py","file_url":"https://github.com/samsungsailmontreal/papa/blob/HEAD/recombine/corr_matching.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"527355ff9a678a4c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}