{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/parameterized-knowledge-transfer-for","title":"Parameterized Knowledge Transfer for Personalized Federated Learning","arxiv_id":"2111.02862","date":"2021-11-04","proceeding":"NeurIPS 2021 12","authors":["Jie Zhang","Song Guo","Xiaosong Ma","Haozhao Wang","Wencao Xu","Feijie Wu"],"abstract":"In recent years, personalized federated learning (pFL) has attracted increasing attention for its potential in dealing with statistical heterogeneity among clients. However, the state-of-the-art pFL methods rely on model parameters aggregation at the server side, which require all models to have the same structure and size, and thus limits the application for more heterogeneous scenarios. To deal with such model constraints, we exploit the potentials of heterogeneous model settings and propose a novel training framework to employ personalized models for different clients. Specifically, we formulate the aggregation procedure in original pFL into a personalized group knowledge transfer training algorithm, namely, KT-pFL, which enables each client to maintain a personalized soft prediction at the server side to guide the others' local training. KT-pFL updates the personalized soft prediction of each client by a linear combination of all local soft predictions using a knowledge coefficient matrix, which can adaptively reinforce the collaboration among clients who own similar data distribution. Furthermore, to quantify the contributions of each client to others' personalized training, the knowledge coefficient matrix is parameterized so that it can be trained simultaneously with the models. The knowledge coefficient matrix and the model parameters are alternatively updated in each round following the gradient descent way. Extensive experiments on various datasets (EMNIST, Fashion\\_MNIST, CIFAR-10) are conducted under different settings (heterogeneous models and data distributions). It is demonstrated that the proposed framework is the first federated learning paradigm that realizes personalized model training via parameterized group knowledge transfer while achieving significant performance gain comparing with state-of-the-art algorithms.","url_abs":"https://arxiv.org/abs/2111.02862v1","url_pdf":"https://arxiv.org/pdf/2111.02862v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"parameterized-knowledge-transfer-for","repo_url":"https://github.com/cugzj/kt-pfl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"personalized-federated-learning","task_name":"Personalized Federated Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.02862","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.02862"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cugzj/kt-pfl","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":1,"ran_honours":1},"by_repo_kind":{"listed":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"5d80d9b1f1823a7f","entry":"KL_loss","repo":"cugzj/kt-pfl","repo_kind":"listed","path":"src/engine_kt_pfl.py","file_url":"https://github.com/cugzj/kt-pfl/blob/HEAD/src/engine_kt_pfl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d80d9b1f1823a7f"}},{"code_sha256_prefix":"def6216db807f469","entry":"SoftCrossEntropy","repo":"cugzj/kt-pfl","repo_kind":"listed","path":"src/engine_kt_pfl.py","file_url":"https://github.com/cugzj/kt-pfl/blob/HEAD/src/engine_kt_pfl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"def6216db807f469"}},{"code_sha256_prefix":"aec9fbc329eb370c","entry":"get_sample_data","repo":"cugzj/kt-pfl","repo_kind":"listed","path":"src/data_utils.py","file_url":"https://github.com/cugzj/kt-pfl/blob/HEAD/src/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aec9fbc329eb370c"}},{"code_sha256_prefix":"4f750a644a725128","entry":"train_one_model","repo":"cugzj/kt-pfl","repo_kind":"listed","path":"src/engine_kt_pfl.py","file_url":"https://github.com/cugzj/kt-pfl/blob/HEAD/src/engine_kt_pfl.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f750a644a725128"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}