{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-learning-via-posterior-averaging-a-1","title":"Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms","arxiv_id":"2010.05273","date":"2020-10-11","proceeding":"ICLR 2021 1","authors":["Maruan Al-Shedivat","Jennifer Gillenwater","Eric Xing","Afshin Rostamizadeh"],"abstract":"Federated learning is typically approached as an optimization problem, where the goal is to minimize a global loss function by distributing computation across client devices that possess local data and specify different parts of the global objective. We present an alternative perspective and formulate federated learning as a posterior inference problem, where the goal is to infer a global posterior distribution by having client devices each infer the posterior of their local data. While exact inference is often intractable, this perspective provides a principled way to search for global optima in federated settings. Further, starting with the analysis of federated quadratic objectives, we develop a computation- and communication-efficient approximate posterior inference algorithm -- federated posterior averaging (FedPA). Our algorithm uses MCMC for approximate inference of local posteriors on the clients and efficiently communicates their statistics to the server, where the latter uses them to refine a global estimate of the posterior mode. Finally, we show that FedPA generalizes federated averaging (FedAvg), can similarly benefit from adaptive optimizers, and yields state-of-the-art results on four realistic and challenging benchmarks, converging faster, to better optima.","url_abs":"https://arxiv.org/abs/2010.05273v4","url_pdf":"https://arxiv.org/pdf/2010.05273v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-learning-via-posterior-averaging-a-1","repo_url":"https://github.com/alshedivat/fedpa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2010.05273","atlas_url":"https://app.syntology.ai/?focus=2010.05273","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.05273"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/alshedivat/fedpa","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/google-research/federated","reach":null}],"summary":{"ran":5,"ran_honours":1,"ran_draft_wrong":3,"ran_fixture":1,"unverified":10},"by_repo_kind":{"official":{"samples":17,"ran":7,"repositories":1},"found_in_text":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"34b0baef8c41a338","entry":"DeltaUpdateOutput","repo":"google-research/federated","repo_kind":"found_in_text","path":"posterior_averaging/fed_pa_schedule.py","file_url":"https://github.com/google-research/federated/blob/HEAD/posterior_averaging/fed_pa_schedule.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"34b0baef8c41a338"}},{"code_sha256_prefix":"9f9c22513d358ef8","entry":"LeastSquares","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9f9c22513d358ef8"}},{"code_sha256_prefix":"8777261d5c9713c4","entry":"ServerState","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8777261d5c9713c4"}},{"code_sha256_prefix":"d181fe7426893bee","entry":"ShrinkageMomentEstimator","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d181fe7426893bee"}},{"code_sha256_prefix":"d2449ec852478290","entry":"Timer","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d2449ec852478290"}},{"code_sha256_prefix":"0a43b5b4b50de92c","entry":"_struct_dot","repo":"google-research/federated","repo_kind":"found_in_text","path":"posterior_averaging/fed_pa_schedule.py","file_url":"https://github.com/google-research/federated/blob/HEAD/posterior_averaging/fed_pa_schedule.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a43b5b4b50de92c"}},{"code_sha256_prefix":"eaa077e6329940db","entry":"_update_delta_posterior_mean","repo":"google-research/federated","repo_kind":"found_in_text","path":"posterior_averaging/fed_pa_schedule.py","file_url":"https://github.com/google-research/federated/blob/HEAD/posterior_averaging/fed_pa_schedule.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eaa077e6329940db"}},{"code_sha256_prefix":"f73b84bbec6b612d","entry":"compute_server_update","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f73b84bbec6b612d"}},{"code_sha256_prefix":"9b2ee5bb98428332","entry":"compute_weighted_average","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9b2ee5bb98428332"}},{"code_sha256_prefix":"e87e32e7160fb616","entry":"sample_clients_uniformly","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e87e32e7160fb616"}},{"code_sha256_prefix":"e5e8feea11b53c2d","entry":"ExactQuadraticSampler","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5e8feea11b53c2d"}},{"code_sha256_prefix":"4122cb4942002b5a","entry":"MomentEstimator","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4122cb4942002b5a"}},{"code_sha256_prefix":"8cf9dfda8fb3ede4","entry":"Objective","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8cf9dfda8fb3ede4"}},{"code_sha256_prefix":"abed1126812891d5","entry":"PosteriorSampler","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"abed1126812891d5"}},{"code_sha256_prefix":"9ba95e935bbdd3a0","entry":"Quadratic","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9ba95e935bbdd3a0"}},{"code_sha256_prefix":"679e078965242bff","entry":"StochasticObjective","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"679e078965242bff"}},{"code_sha256_prefix":"e40f2ebbac79d228","entry":"compute_post_avg_delta","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e40f2ebbac79d228"}},{"code_sha256_prefix":"59a6c293bce93c9b","entry":"compute_post_avg_delta_dp","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"59a6c293bce93c9b"}},{"code_sha256_prefix":"9384f324dbb90599","entry":"create_post_avg_exact","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9384f324dbb90599"}},{"code_sha256_prefix":"165dada095da4159","entry":"fed_opt","repo":"alshedivat/fedpa","repo_kind":"official","path":"federated/learning/algorithms.py","file_url":"https://github.com/alshedivat/fedpa/blob/HEAD/federated/learning/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"165dada095da4159"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}