{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spectral-co-distillation-for-personalized-1","title":"Spectral Co-Distillation for Personalized Federated Learning","arxiv_id":"2401.17124","date":"2024-01-29","proceeding":"NeurIPS 2023 11","authors":["Zihan Chen","Howard H. Yang","Tony Q. S. Quek","Kai Fong Ernest Chong"],"abstract":"Personalized federated learning (PFL) has been widely investigated to address the challenge of data heterogeneity, especially when a single generic model is inadequate in satisfying the diverse performance requirements of local clients simultaneously. Existing PFL methods are inherently based on the idea that the relations between the generic global and personalized local models are captured by the similarity of model weights. Such a similarity is primarily based on either partitioning the model architecture into generic versus personalized components, or modeling client relationships via model weights. To better capture similar (yet distinct) generic versus personalized model representations, we propose \\textit{spectral distillation}, a novel distillation method based on model spectrum information. Building upon spectral distillation, we also introduce a co-distillation framework that establishes a two-way bridge between generic and personalized model training. Moreover, to utilize the local idle time in conventional PFL, we propose a wait-free local training protocol. Through extensive experiments on multiple datasets over diverse heterogeneous data settings, we demonstrate the outperformance and efficacy of our proposed spectral co-distillation method, as well as our wait-free training protocol.","url_abs":"https://arxiv.org/abs/2401.17124v1","url_pdf":"https://arxiv.org/pdf/2401.17124v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spectral-co-distillation-for-personalized-1","repo_url":"https://github.com/jimmyc96/spectral-dis-fl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"personalized-federated-learning","task_name":"Personalized Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.17124","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.17124"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jimmyc96/spectral-dis-fl","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"0d53a550f390dfd0","entry":"FFT_weights_cal","repo":"jimmyc96/spectral-dis-fl","repo_kind":"official","path":"util/update.py","file_url":"https://github.com/jimmyc96/spectral-dis-fl/blob/HEAD/util/update.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d53a550f390dfd0"}},{"code_sha256_prefix":"e96e3067a91696d1","entry":"compute_fft_of_weights","repo":"jimmyc96/spectral-dis-fl","repo_kind":"official","path":"util/update.py","file_url":"https://github.com/jimmyc96/spectral-dis-fl/blob/HEAD/util/update.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e96e3067a91696d1"}},{"code_sha256_prefix":"01f572f7d085ffd9","entry":"spectral_cal","repo":"jimmyc96/spectral-dis-fl","repo_kind":"official","path":"util/update.py","file_url":"https://github.com/jimmyc96/spectral-dis-fl/blob/HEAD/util/update.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01f572f7d085ffd9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}