{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gpt-fl-generative-pre-trained-model-assisted","title":"GPT-FL: Generative Pre-trained Model-Assisted Federated Learning","arxiv_id":"2306.02210","date":"2023-06-03","proceeding":null,"authors":["Tuo Zhang","Tiantian Feng","Samiul Alam","Dimitrios Dimitriadis","Sunwoo Lee","Mi Zhang","Shrikanth S. Narayanan","Salman Avestimehr"],"abstract":"In this work, we propose GPT-FL, a generative pre-trained model-assisted federated learning (FL) framework. At its core, GPT-FL leverages generative pre-trained models to generate diversified synthetic data. These generated data are used to train a downstream model on the server, which is then fine-tuned with private client data under the standard FL framework. We show that GPT-FL consistently outperforms state-of-the-art FL methods in terms of model test accuracy, communication efficiency, and client sampling efficiency. Through comprehensive ablation analysis across various data modalities, we discover that the downstream model generated by synthetic data plays a crucial role in controlling the direction of gradient diversity during FL training, which enhances convergence speed and contributes to the notable accuracy boost observed with GPT-FL. Also, regardless of whether the target data falls within or outside the domain of the pre-trained generative model, GPT-FL consistently achieves significant performance gains, surpassing the results obtained by models trained solely with FL or synthetic data. The code is available at https://github.com/AvestimehrResearchGroup/GPT-FL.","url_abs":"https://arxiv.org/abs/2306.02210v4","url_pdf":"https://arxiv.org/pdf/2306.02210v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gpt-fl-generative-pre-trained-model-assisted","repo_url":"https://github.com/AvestimehrResearchGroup/GPT-FL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.02210","atlas_url":"https://app.syntology.ai/?focus=2306.02210","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02210"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AvestimehrResearchGroup/GPT-FL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"661319e0af1646dc","entry":"get_model","repo":"AvestimehrResearchGroup/GPT-FL","repo_kind":"official","path":"synthetic_data_generation/convnet.py","file_url":"https://github.com/AvestimehrResearchGroup/GPT-FL/blob/HEAD/synthetic_data_generation/convnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"661319e0af1646dc"}},{"code_sha256_prefix":"31d0fab700c6e892","entry":"get_model","repo":"AvestimehrResearchGroup/GPT-FL","repo_kind":"official","path":"EasyFL/model/convnet.py","file_url":"https://github.com/AvestimehrResearchGroup/GPT-FL/blob/HEAD/EasyFL/model/convnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"31d0fab700c6e892"}},{"code_sha256_prefix":"e35a16902bc77d13","entry":"mobilenetv2","repo":"AvestimehrResearchGroup/GPT-FL","repo_kind":"official","path":"synthetic_data_generation/convnet.py","file_url":"https://github.com/AvestimehrResearchGroup/GPT-FL/blob/HEAD/synthetic_data_generation/convnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"e35a16902bc77d13"}},{"code_sha256_prefix":"140905d48b761a27","entry":"resnet20","repo":"AvestimehrResearchGroup/GPT-FL","repo_kind":"official","path":"synthetic_data_generation/resnet.py","file_url":"https://github.com/AvestimehrResearchGroup/GPT-FL/blob/HEAD/synthetic_data_generation/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"140905d48b761a27"}},{"code_sha256_prefix":"e6d2849e622ef725","entry":"resnet32","repo":"AvestimehrResearchGroup/GPT-FL","repo_kind":"official","path":"synthetic_data_generation/resnet.py","file_url":"https://github.com/AvestimehrResearchGroup/GPT-FL/blob/HEAD/synthetic_data_generation/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"e6d2849e622ef725"}},{"code_sha256_prefix":"45360a9cb0aa934f","entry":"resnet44","repo":"AvestimehrResearchGroup/GPT-FL","repo_kind":"official","path":"synthetic_data_generation/resnet.py","file_url":"https://github.com/AvestimehrResearchGroup/GPT-FL/blob/HEAD/synthetic_data_generation/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"45360a9cb0aa934f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}