{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-learning-based-on-dynamic-1","title":"Federated Learning Based on Dynamic Regularization","arxiv_id":"2111.04263","date":"2021-11-08","proceeding":"ICLR 2021 1","authors":["Durmus Alp Emre Acar","Yue Zhao","Ramon Matas Navarro","Matthew Mattina","Paul N. Whatmough","Venkatesh Saligrama"],"abstract":"We propose a novel federated learning method for distributively training neural network models, where the server orchestrates cooperation between a subset of randomly chosen devices in each round. We view Federated Learning problem primarily from a communication perspective and allow more device level computations to save transmission costs. We point out a fundamental dilemma, in that the minima of the local-device level empirical loss are inconsistent with those of the global empirical loss. Different from recent prior works, that either attempt inexact minimization or utilize devices for parallelizing gradient computation, we propose a dynamic regularizer for each device at each round, so that in the limit the global and device solutions are aligned. We demonstrate both through empirical results on real and synthetic data as well as analytical results that our scheme leads to efficient training, in both convex and non-convex settings, while being fully agnostic to device heterogeneity and robust to large number of devices, partial participation and unbalanced data.","url_abs":"https://arxiv.org/abs/2111.04263v2","url_pdf":"https://arxiv.org/pdf/2111.04263v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-learning-based-on-dynamic-1","repo_url":"https://github.com/alpemreacar/FedDyn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"federated-learning-based-on-dynamic-1","repo_url":"https://github.com/KarhouTam/FL-bench","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"federated-learning-based-on-dynamic-1","repo_url":"https://github.com/NAVER-INTEL-Co-Lab/gaudi-byzantine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"federated-learning-based-on-dynamic-1","repo_url":"https://github.com/thejungwon/gc-fed","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"federated-learning-based-on-dynamic-1","repo_url":"https://github.com/TsingZ0/PFL-Non-IID","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.04263","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.04263"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alpemreacar/FedDyn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thejungwon/gc-fed","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TsingZ0/PFL-Non-IID","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NAVER-INTEL-Co-Lab/gaudi-byzantine","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KarhouTam/FL-bench","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran":4,"ran_honours":1,"ran_draft_wrong":1,"unverified":9},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1},"listed":{"samples":14,"ran":6,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"cff80b8ecb8c8c99","entry":"CNN","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cff80b8ecb8c8c99"}},{"code_sha256_prefix":"acd09ebeda2c9bc6","entry":"MLP","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"acd09ebeda2c9bc6"}},{"code_sha256_prefix":"ce17439507a51476","entry":"ResNet18","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce17439507a51476"}},{"code_sha256_prefix":"f0aeb868a782b3d7","entry":"VGG11","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0aeb868a782b3d7"}},{"code_sha256_prefix":"1f7693bbd3b8865f","entry":"model_parameter_vector","repo":"TsingZ0/PFL-Non-IID","repo_kind":"listed","path":"system/flcore/servers/serverdyn.py","file_url":"https://github.com/TsingZ0/PFL-Non-IID/blob/HEAD/system/flcore/servers/serverdyn.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f7693bbd3b8865f"}},{"code_sha256_prefix":"a0b65033d450bccb","entry":"server_dyn","repo":"NAVER-INTEL-Co-Lab/gaudi-byzantine","repo_kind":"listed","path":"src/aggregation.py","file_url":"https://github.com/NAVER-INTEL-Co-Lab/gaudi-byzantine/blob/HEAD/src/aggregation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0b65033d450bccb"}},{"code_sha256_prefix":"a327f364bcb74163","entry":"FedDyn","repo":"TsingZ0/PFL-Non-IID","repo_kind":"listed","path":"system/flcore/servers/serverdyn.py","file_url":"https://github.com/TsingZ0/PFL-Non-IID/blob/HEAD/system/flcore/servers/serverdyn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a327f364bcb74163"}},{"code_sha256_prefix":"70803d2f07b42d4a","entry":"GCFedClient","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70803d2f07b42d4a"}},{"code_sha256_prefix":"0a6af52fc93a95a1","entry":"Server","repo":"TsingZ0/PFL-Non-IID","repo_kind":"listed","path":"system/flcore/servers/serverdyn.py","file_url":"https://github.com/TsingZ0/PFL-Non-IID/blob/HEAD/system/flcore/servers/serverdyn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0a6af52fc93a95a1"}},{"code_sha256_prefix":"16480f2cf0b819dd","entry":"clientDyn","repo":"TsingZ0/PFL-Non-IID","repo_kind":"listed","path":"system/flcore/servers/serverdyn.py","file_url":"https://github.com/TsingZ0/PFL-Non-IID/blob/HEAD/system/flcore/servers/serverdyn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"16480f2cf0b819dd"}},{"code_sha256_prefix":"021173b9b0b6da38","entry":"get_model","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"021173b9b0b6da38"}},{"code_sha256_prefix":"39c6548f5ca70964","entry":"init_weights","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39c6548f5ca70964"}},{"code_sha256_prefix":"a2a34e406dfeba02","entry":"log_info","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a2a34e406dfeba02"}},{"code_sha256_prefix":"c08e488e6cf86c88","entry":"set_random_seed","repo":"thejungwon/gc-fed","repo_kind":"listed","path":"algorithms/gcfed.py","file_url":"https://github.com/thejungwon/gc-fed/blob/HEAD/algorithms/gcfed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c08e488e6cf86c88"}},{"code_sha256_prefix":"d8e5236ae622abdc","entry":"train_FedDyn","repo":"alpemreacar/FedDyn","repo_kind":"official","path":"utils_methods.py","file_url":"https://github.com/alpemreacar/FedDyn/blob/HEAD/utils_methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d8e5236ae622abdc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}