{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decoupling-general-and-personalized-knowledge","title":"Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decomposition","arxiv_id":"2406.19931","date":"2024-06-28","proceeding":null,"authors":["Xinghao Wu","Xuefeng Liu","Jianwei Niu","Haolin Wang","Shaojie Tang","Guogang Zhu","Hao Su"],"abstract":"To address data heterogeneity, the key strategy of Personalized Federated Learning (PFL) is to decouple general knowledge (shared among clients) and client-specific knowledge, as the latter can have a negative impact on collaboration if not removed. Existing PFL methods primarily adopt a parameter partitioning approach, where the parameters of a model are designated as one of two types: parameters shared with other clients to extract general knowledge and parameters retained locally to learn client-specific knowledge. However, as these two types of parameters are put together like a jigsaw puzzle into a single model during the training process, each parameter may simultaneously absorb both general and client-specific knowledge, thus struggling to separate the two types of knowledge effectively. In this paper, we introduce FedDecomp, a simple but effective PFL paradigm that employs parameter additive decomposition to address this issue. Instead of assigning each parameter of a model as either a shared or personalized one, FedDecomp decomposes each parameter into the sum of two parameters: a shared one and a personalized one, thus achieving a more thorough decoupling of shared and personalized knowledge compared to the parameter partitioning method. In addition, as we find that retaining local knowledge of specific clients requires much lower model capacity compared with general knowledge across all clients, we let the matrix containing personalized parameters be low rank during the training process. Moreover, a new alternating training strategy is proposed to further improve the performance. Experimental results across multiple datasets and varying degrees of data heterogeneity demonstrate that FedDecomp outperforms state-of-the-art methods up to 4.9\\%. The code is available at https://github.com/XinghaoWu/FedDecomp.","url_abs":"https://arxiv.org/abs/2406.19931v2","url_pdf":"https://arxiv.org/pdf/2406.19931v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decoupling-general-and-personalized-knowledge","repo_url":"https://github.com/xinghaowu/feddecomp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"general-knowledge","task_name":"General Knowledge"},{"task_slug":"personalized-federated-learning","task_name":"Personalized Federated Learning"}],"methods":[{"method_slug":"jigsaw","method_name":"Jigsaw"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.19931","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.19931"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xinghaowu/feddecomp","reach":{"status":"ok"}}],"summary":{"ran":4,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"9e794f046561a69a","entry":"FedAvg","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"models/Fed.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/models/Fed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e794f046561a69a"}},{"code_sha256_prefix":"2a80220dabcb742a","entry":"conv1x1","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a80220dabcb742a"}},{"code_sha256_prefix":"600ff2c45e0de056","entry":"conv3x3","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"600ff2c45e0de056"}},{"code_sha256_prefix":"660073ae6862ac6e","entry":"lora_state_dict","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"loralib/utils.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/loralib/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"660073ae6862ac6e"}},{"code_sha256_prefix":"84deaace150221ec","entry":"mnist_iid","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"utils/sampling.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"84deaace150221ec"}},{"code_sha256_prefix":"f5c4e6e06e4da209","entry":"pathological_noniid","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"utils/sampling.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f5c4e6e06e4da209"}},{"code_sha256_prefix":"ed298f589e1b0a38","entry":"dirichlet_noniid","repo":"xinghaowu/feddecomp","repo_kind":"official","path":"utils/sampling.py","file_url":"https://github.com/xinghaowu/feddecomp/blob/HEAD/utils/sampling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed298f589e1b0a38"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}