{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-learning-from-pre-trained-models-a","title":"Federated Learning from Pre-Trained Models: A Contrastive Learning Approach","arxiv_id":"2209.10083","date":"2022-09-21","proceeding":null,"authors":["Yue Tan","Guodong Long","Jie Ma","Lu Liu","Tianyi Zhou","Jing Jiang"],"abstract":"Federated Learning (FL) is a machine learning paradigm that allows decentralized clients to learn collaboratively without sharing their private data. However, excessive computation and communication demands pose challenges to current FL frameworks, especially when training large-scale models. To prevent these issues from hindering the deployment of FL systems, we propose a lightweight framework where clients jointly learn to fuse the representations generated by multiple fixed pre-trained models rather than training a large-scale model from scratch. This leads us to a more practical FL problem by considering how to capture more client-specific and class-relevant information from the pre-trained models and jointly improve each client's ability to exploit those off-the-shelf models. In this work, we design a Federated Prototype-wise Contrastive Learning (FedPCL) approach which shares knowledge across clients through their class prototypes and builds client-specific representations in a prototype-wise contrastive manner. Sharing prototypes rather than learnable model parameters allows each client to fuse the representations in a personalized way while keeping the shared knowledge in a compact form for efficient communication. We perform a thorough evaluation of the proposed FedPCL in the lightweight framework, measuring and visualizing its ability to fuse various pre-trained models on popular FL datasets.","url_abs":"https://arxiv.org/abs/2209.10083v1","url_pdf":"https://arxiv.org/pdf/2209.10083v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-learning-from-pre-trained-models-a","repo_url":"https://github.com/TsingZ0/GFL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"federated-learning-from-pre-trained-models-a","repo_url":"https://github.com/TsingZ0/PFL-Non-IID","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.10083","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.10083"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TsingZ0/PFL-Non-IID","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TsingZ0/GFL","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran":2,"ran_honours":1,"unverified":1},"by_repo_kind":{"listed":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5d6dc231986a939a","entry":"agg_func","repo":"TsingZ0/GFL","repo_kind":"listed","path":"system/flcore/clients/clientktl.py","file_url":"https://github.com/TsingZ0/GFL/blob/HEAD/system/flcore/clients/clientktl.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d6dc231986a939a"}},{"code_sha256_prefix":"34ebb1820a560b2b","entry":"agg_func","repo":"TsingZ0/GFL","repo_kind":"listed","path":"system/flcore/clients/clientgh.py","file_url":"https://github.com/TsingZ0/GFL/blob/HEAD/system/flcore/clients/clientgh.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"34ebb1820a560b2b"}},{"code_sha256_prefix":"00a3ba9dbf0b838b","entry":"agg_func","repo":"TsingZ0/GFL","repo_kind":"listed","path":"system/flcore/clients/clientfd.py","file_url":"https://github.com/TsingZ0/GFL/blob/HEAD/system/flcore/clients/clientfd.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00a3ba9dbf0b838b"}},{"code_sha256_prefix":"a296eeb8fbc6ef9e","entry":"decomposition","repo":"TsingZ0/GFL","repo_kind":"listed","path":"system/flcore/clients/clientkd.py","file_url":"https://github.com/TsingZ0/GFL/blob/HEAD/system/flcore/clients/clientkd.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a296eeb8fbc6ef9e"}},{"code_sha256_prefix":"132dfc2af9ff530c","entry":"load_item","repo":"TsingZ0/GFL","repo_kind":"listed","path":"system/flcore/clients/clientbase.py","file_url":"https://github.com/TsingZ0/GFL/blob/HEAD/system/flcore/clients/clientbase.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"132dfc2af9ff530c"}},{"code_sha256_prefix":"77fd3aefd63cfad0","entry":"recover","repo":"TsingZ0/GFL","repo_kind":"listed","path":"system/flcore/clients/clientkd.py","file_url":"https://github.com/TsingZ0/GFL/blob/HEAD/system/flcore/clients/clientkd.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"77fd3aefd63cfad0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}