{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/feddistill-making-bayesian-model-ensemble","title":"FedBE: Making Bayesian Model Ensemble Applicable to Federated Learning","arxiv_id":"2009.01974","date":"2020-09-04","proceeding":"ICLR 2021 1","authors":["Hong-You Chen","Wei-Lun Chao"],"abstract":"Federated learning aims to collaboratively train a strong global model by accessing users' locally trained models but not their own data. A crucial step is therefore to aggregate local models into a global model, which has been shown challenging when users have non-i.i.d. data. In this paper, we propose a novel aggregation algorithm named FedBE, which takes a Bayesian inference perspective by sampling higher-quality global models and combining them via Bayesian model Ensemble, leading to much robust aggregation. We show that an effective model distribution can be constructed by simply fitting a Gaussian or Dirichlet distribution to the local models. Our empirical studies validate FedBE's superior performance, especially when users' data are not i.i.d. and when the neural networks go deeper. Moreover, FedBE is compatible with recent efforts in regularizing users' model training, making it an easily applicable module: you only need to replace the aggregation method but leave other parts of your federated learning algorithm intact. Our code is publicly available at https://github.com/hongyouc/FedBE.","url_abs":"https://arxiv.org/abs/2009.01974v4","url_pdf":"https://arxiv.org/pdf/2009.01974v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"feddistill-making-bayesian-model-ensemble","repo_url":"https://github.com/hongyouc/fedbe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"feddistill-making-bayesian-model-ensemble","repo_url":"https://github.com/sabhayjain/FedBE-TPMI-2021","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2009.01974","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2009.01974"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hongyouc/FedBE","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hongyouc/fedbe","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sabhayjain/FedBE-TPMI-2021","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"276295afe2705ae2","entry":"FedAvg","repo":"hongyouc/FedBE","repo_kind":"official","path":"models/Fed.py","file_url":"https://github.com/hongyouc/FedBE/blob/HEAD/models/Fed.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"276295afe2705ae2"}},{"code_sha256_prefix":"a672de245adafb60","entry":"create_local_init","repo":"hongyouc/FedBE","repo_kind":"official","path":"models/Fed.py","file_url":"https://github.com/hongyouc/FedBE/blob/HEAD/models/Fed.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a672de245adafb60"}},{"code_sha256_prefix":"d9107114749c3e49","entry":"mnist_iid","repo":"hongyouc/FedBE","repo_kind":"official","path":"utils/sampling.py","file_url":"https://github.com/hongyouc/FedBE/blob/HEAD/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9107114749c3e49"}},{"code_sha256_prefix":"d9d721ef1e522392","entry":"mnist_noniid","repo":"hongyouc/FedBE","repo_kind":"official","path":"utils/sampling.py","file_url":"https://github.com/hongyouc/FedBE/blob/HEAD/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9d721ef1e522392"}},{"code_sha256_prefix":"3ed041e66ac061a8","entry":"SWAG_server","repo":"hongyouc/fedbe","repo_kind":"official","path":"swag.py","file_url":"https://github.com/hongyouc/fedbe/blob/HEAD/swag.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3ed041e66ac061a8"}},{"code_sha256_prefix":"8aeca3fd6a73c15f","entry":"cifar_iid","repo":"hongyouc/FedBE","repo_kind":"official","path":"utils/sampling.py","file_url":"https://github.com/hongyouc/FedBE/blob/HEAD/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8aeca3fd6a73c15f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}