{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaffold-stochastic-controlled-averaging-for","title":"SCAFFOLD: Stochastic Controlled Averaging for Federated Learning","arxiv_id":"1910.06378","date":"2019-10-14","proceeding":"ICML 2020 1","authors":["Sai Praneeth Karimireddy","Satyen Kale","Mehryar Mohri","Sashank J. Reddi","Sebastian U. Stich","Ananda Theertha Suresh"],"abstract":"Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully understood. We obtain tight convergence rates for FedAvg and prove that it suffers from `client-drift' when the data is heterogeneous (non-iid), resulting in unstable and slow convergence. As a solution, we propose a new algorithm (SCAFFOLD) which uses control variates (variance reduction) to correct for the `client-drift' in its local updates. We prove that SCAFFOLD requires significantly fewer communication rounds and is not affected by data heterogeneity or client sampling. Further, we show that (for quadratics) SCAFFOLD can take advantage of similarity in the client's data yielding even faster convergence. The latter is the first result to quantify the usefulness of local-steps in distributed optimization.","url_abs":"https://arxiv.org/abs/1910.06378v4","url_pdf":"https://arxiv.org/pdf/1910.06378v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/KarhouTam/FL-bench","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/KarhouTam/SCAFFOLD-PyTorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/carbonati/fl-zoo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/google-research/public-data-in-dpfl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/ramshi236/Accelerated-Federated-Learning-Over-MAC-in-Heterogeneous-Networks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/thejungwon/gc-fed","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/TsingZ0/PFL-Non-IID","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"scaffold-stochastic-controlled-averaging-for","repo_url":"https://github.com/ki-ljl/Scaffold-Federated-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"distributed-optimization","task_name":"Distributed Optimization"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.06378","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1910.06378"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/carbonati/fl-zoo","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thejungwon/gc-fed","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TsingZ0/PFL-Non-IID","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/public-data-in-dpfl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KarhouTam/FL-bench","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ki-ljl/Scaffold-Federated-Learning","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ramshi236/Accelerated-Federated-Learning-Over-MAC-in-Heterogeneous-Networks","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KarhouTam/SCAFFOLD-PyTorch","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"listed":{"samples":4,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3643bd05b1da6428","entry":"clone_parameters","repo":"KarhouTam/SCAFFOLD-PyTorch","repo_kind":"listed","path":"src/config/util.py","file_url":"https://github.com/KarhouTam/SCAFFOLD-PyTorch/blob/HEAD/src/config/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3643bd05b1da6428"}},{"code_sha256_prefix":"bd47eaab015e63fe","entry":"get_mape","repo":"ki-ljl/Scaffold-Federated-Learning","repo_kind":"listed","path":"get_data.py","file_url":"https://github.com/ki-ljl/Scaffold-Federated-Learning/blob/HEAD/get_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd47eaab015e63fe"}},{"code_sha256_prefix":"4a50bd68763f2dc6","entry":"load_data","repo":"ki-ljl/Scaffold-Federated-Learning","repo_kind":"listed","path":"get_data.py","file_url":"https://github.com/ki-ljl/Scaffold-Federated-Learning/blob/HEAD/get_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a50bd68763f2dc6"}},{"code_sha256_prefix":"64f7bd048cc61b61","entry":"nn_seq_wind","repo":"ki-ljl/Scaffold-Federated-Learning","repo_kind":"listed","path":"get_data.py","file_url":"https://github.com/ki-ljl/Scaffold-Federated-Learning/blob/HEAD/get_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64f7bd048cc61b61"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}