{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dual-personalizing-adapter-for-federated","title":"Dual-Personalizing Adapter for Federated Foundation Models","arxiv_id":"2403.19211","date":"2024-03-28","proceeding":null,"authors":["Yiyuan Yang","Guodong Long","Tao Shen","Jing Jiang","Michael Blumenstein"],"abstract":"Recently, foundation models, particularly large language models (LLMs), have demonstrated an impressive ability to adapt to various tasks by fine-tuning diverse instruction data. Notably, federated foundation models (FedFM) emerge as a privacy preservation method to fine-tune models collaboratively under federated learning (FL) settings by leveraging many distributed datasets with non-IID data. To alleviate communication and computation overhead, parameter-efficient methods are introduced for efficiency, and some research adapted personalization methods to FedFM for better user preferences alignment. However, a critical gap in existing research is the neglect of test-time distribution shifts in real-world applications, and conventional methods for test-time distribution shifts in personalized FL are less effective for FedFM due to their failure to adapt to complex distribution shift scenarios and the requirement to train all parameters. To bridge this gap, we refine the setting in FedFM, termed test-time personalization, which aims to learn personalized federated foundation models on clients while effectively handling test-time distribution shifts simultaneously. To address challenges in this setting, we explore a simple yet effective solution, a Federated Dual-Personalizing Adapter (FedDPA) architecture. By co-working with a foundation model, a global adapter and a local adapter jointly tackle the test-time distribution shifts and client-specific personalization. Additionally, we introduce an instance-wise dynamic weighting mechanism that dynamically integrates the global and local adapters for each test instance during inference, facilitating effective test-time personalization. The effectiveness of the proposed method has been evaluated on benchmark datasets across different NLP tasks.","url_abs":"https://arxiv.org/abs/2403.19211v2","url_pdf":"https://arxiv.org/pdf/2403.19211v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dual-personalizing-adapter-for-federated","repo_url":"https://github.com/Lydia-yang/FedDPA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2403.19211","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.19211"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Lydia-yang/FedDPA","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"7d997ff019c2baf7","entry":"client_selection","repo":"Lydia-yang/FedDPA","repo_kind":"official","path":"code/fed_utils/client_participation_scheduling.py","file_url":"https://github.com/Lydia-yang/FedDPA/blob/HEAD/code/fed_utils/client_participation_scheduling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d997ff019c2baf7"}},{"code_sha256_prefix":"764c287671e2023f","entry":"read_list","repo":"Lydia-yang/FedDPA","repo_kind":"official","path":"code/metric.py","file_url":"https://github.com/Lydia-yang/FedDPA/blob/HEAD/code/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"764c287671e2023f"}},{"code_sha256_prefix":"5826791292da68e9","entry":"FedAvg","repo":"Lydia-yang/FedDPA","repo_kind":"official","path":"code/fed_utils/model_aggregation.py","file_url":"https://github.com/Lydia-yang/FedDPA/blob/HEAD/code/fed_utils/model_aggregation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5826791292da68e9"}},{"code_sha256_prefix":"eb118c3a5abf1380","entry":"global_evaluation","repo":"Lydia-yang/FedDPA","repo_kind":"official","path":"code/fed_utils/evaluation.py","file_url":"https://github.com/Lydia-yang/FedDPA/blob/HEAD/code/fed_utils/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb118c3a5abf1380"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}