{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hierarchical-federated-learning-with-multi","title":"Hierarchical Federated Learning with Multi-Timescale Gradient Correction","arxiv_id":"2409.18448","date":"2024-09-27","proceeding":null,"authors":["Wenzhi Fang","Dong-Jun Han","Evan Chen","Shiqiang Wang","Christopher G. Brinton"],"abstract":"While traditional federated learning (FL) typically focuses on a star topology where clients are directly connected to a central server, real-world distributed systems often exhibit hierarchical architectures. Hierarchical FL (HFL) has emerged as a promising solution to bridge this gap, leveraging aggregation points at multiple levels of the system. However, existing algorithms for HFL encounter challenges in dealing with multi-timescale model drift, i.e., model drift occurring across hierarchical levels of data heterogeneity. In this paper, we propose a multi-timescale gradient correction (MTGC) methodology to resolve this issue. Our key idea is to introduce distinct control variables to (i) correct the client gradient towards the group gradient, i.e., to reduce client model drift caused by local updates based on individual datasets, and (ii) correct the group gradient towards the global gradient, i.e., to reduce group model drift caused by FL over clients within the group. We analytically characterize the convergence behavior of MTGC under general non-convex settings, overcoming challenges associated with couplings between correction terms. We show that our convergence bound is immune to the extent of data heterogeneity, confirming the stability of the proposed algorithm against multi-level non-i.i.d. data. Through extensive experiments on various datasets and models, we validate the effectiveness of MTGC in diverse HFL settings. The code for this project is available at \\href{https://github.com/wenzhifang/MTGC}{https://github.com/wenzhifang/MTGC}.","url_abs":"https://arxiv.org/abs/2409.18448v3","url_pdf":"https://arxiv.org/pdf/2409.18448v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hierarchical-federated-learning-with-multi","repo_url":"https://github.com/wenzhifang/mtgc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.18448","atlas_url":"https://app.syntology.ai/?focus=2409.18448","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.18448"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/wenzhifang/MTGC","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wenzhifang/mtgc","reach":{"status":"ok"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"9b9108a92d0c7093","entry":"get_mdl_params","repo":"wenzhifang/MTGC","repo_kind":"official","path":"MTGC_Implementation/utils_general.py","file_url":"https://github.com/wenzhifang/MTGC/blob/HEAD/MTGC_Implementation/utils_general.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b9108a92d0c7093"}},{"code_sha256_prefix":"3c5817b8701c0fd8","entry":"train_FedAvg","repo":"wenzhifang/MTGC","repo_kind":"official","path":"MTGC_Implementation/utils_methods.py","file_url":"https://github.com/wenzhifang/MTGC/blob/HEAD/MTGC_Implementation/utils_methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c5817b8701c0fd8"}},{"code_sha256_prefix":"42f08dded2da3002","entry":"train_MTGC","repo":"wenzhifang/mtgc","repo_kind":"official","path":"MTGC_Implementation/utils_methods.py","file_url":"https://github.com/wenzhifang/mtgc/blob/HEAD/MTGC_Implementation/utils_methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42f08dded2da3002"}},{"code_sha256_prefix":"17bef7bd2ed8b2e3","entry":"train_MTGC_Z","repo":"wenzhifang/MTGC","repo_kind":"official","path":"MTGC_Implementation/utils_methods.py","file_url":"https://github.com/wenzhifang/MTGC/blob/HEAD/MTGC_Implementation/utils_methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17bef7bd2ed8b2e3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}