{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/federated-multi-task-learning-under-a-mixture","title":"Federated Multi-Task Learning under a Mixture of Distributions","arxiv_id":"2108.10252","date":"2021-08-23","proceeding":"NeurIPS 2021 12","authors":["Othmane Marfoq","Giovanni Neglia","Aurélien Bellet","Laetitia Kameni","Richard Vidal"],"abstract":"The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models. First efforts in FL focused on learning a single global model with good average performance across clients, but the global model may be arbitrarily bad for a given client, due to the inherent heterogeneity of local data distributions. Federated multi-task learning (MTL) approaches can learn personalized models by formulating an opportune penalized optimization problem. The penalization term can capture complex relations among personalized models, but eschews clear statistical assumptions about local data distributions. In this work, we propose to study federated MTL under the flexible assumption that each local data distribution is a mixture of unknown underlying distributions. This assumption encompasses most of the existing personalized FL approaches and leads to federated EM-like algorithms for both client-server and fully decentralized settings. Moreover, it provides a principled way to serve personalized models to clients not seen at training time. The algorithms' convergence is analyzed through a novel federated surrogate optimization framework, which can be of general interest. Experimental results on FL benchmarks show that our approach provides models with higher accuracy and fairness than state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2108.10252v4","url_pdf":"https://arxiv.org/pdf/2108.10252v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"federated-multi-task-learning-under-a-mixture","repo_url":"https://github.com/omarfoq/fedem","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"federated-multi-task-learning-under-a-mixture","repo_url":"https://github.com/KarhouTam/FL-bench","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"federated-multi-task-learning-under-a-mixture","repo_url":"https://github.com/bdemo/pfedbred_public","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"federated-multi-task-learning-under-a-mixture","repo_url":"https://github.com/tj-kim/pfeddef_v1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"federated-multi-task-learning-under-a-mixture","repo_url":"https://github.com/yixinyang69/newfedem","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"personalized-federated-learning","task_name":"Personalized Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.10252","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.10252"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/omarfoq/FedEM","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bdemo/pfedbred_public","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yixinyang69/newfedem","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KarhouTam/FL-bench","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/omarfoq/fedem","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tj-kim/pfeddef_v1","reach":null}],"summary":{"ran_fixture":1,"ran_violates":1,"ran_draft_wrong":1,"unverified":14},"by_repo_kind":{"official":{"samples":14,"ran":0,"repositories":1},"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"558fdb5c17a3a411","entry":"cuda","repo":"tj-kim/pfeddef_v1","repo_kind":"listed","path":"transfer_attacks/TA_utils.py","file_url":"https://github.com/tj-kim/pfeddef_v1/blob/HEAD/transfer_attacks/TA_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"558fdb5c17a3a411"}},{"code_sha256_prefix":"f017532fc389cbfe","entry":"str2bool","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f017532fc389cbfe"}},{"code_sha256_prefix":"f5f8985f740c045e","entry":"where","repo":"tj-kim/pfeddef_v1","repo_kind":"listed","path":"transfer_attacks/TA_utils.py","file_url":"https://github.com/tj-kim/pfeddef_v1/blob/HEAD/transfer_attacks/TA_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f5f8985f740c045e"}},{"code_sha256_prefix":"1484765c5a875320","entry":"accuracy","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/metrics.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1484765c5a875320"}},{"code_sha256_prefix":"2aa9eaee30e6f00f","entry":"args_to_string","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/args.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/args.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2aa9eaee30e6f00f"}},{"code_sha256_prefix":"d86b1b7917839f9d","entry":"binary_accuracy","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/metrics.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d86b1b7917839f9d"}},{"code_sha256_prefix":"063c0646ab2e0dd5","entry":"compute_mixing_matrix","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/decentralized.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/decentralized.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"063c0646ab2e0dd5"}},{"code_sha256_prefix":"90e04d573188fcbd","entry":"get_communication_graph","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/decentralized.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/decentralized.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"90e04d573188fcbd"}},{"code_sha256_prefix":"2581454f884675e8","entry":"get_lr_scheduler","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/optim.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/optim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2581454f884675e8"}},{"code_sha256_prefix":"b7accb5913464f4a","entry":"get_mixing_matrix","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/decentralized.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/decentralized.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b7accb5913464f4a"}},{"code_sha256_prefix":"9c1e58d7bca6b6c9","entry":"get_mobilenet","repo":"omarfoq/FedEM","repo_kind":"official","path":"models.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c1e58d7bca6b6c9"}},{"code_sha256_prefix":"2131ebbda92151a1","entry":"get_optimizer","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/optim.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/optim.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2131ebbda92151a1"}},{"code_sha256_prefix":"3a84a16cfdc372ba","entry":"get_squeezenet","repo":"omarfoq/FedEM","repo_kind":"official","path":"models.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3a84a16cfdc372ba"}},{"code_sha256_prefix":"2d5ef5672e97f155","entry":"get_vgg11","repo":"omarfoq/FedEM","repo_kind":"official","path":"models.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d5ef5672e97f155"}},{"code_sha256_prefix":"825bbd029f466ff8","entry":"init_clients","repo":"omarfoq/FedEM","repo_kind":"official","path":"run_experiment.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"825bbd029f466ff8"}},{"code_sha256_prefix":"e5d7e349519111ef","entry":"mse","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/metrics.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5d7e349519111ef"}},{"code_sha256_prefix":"214218b6675ec25e","entry":"parse_args","repo":"omarfoq/FedEM","repo_kind":"official","path":"utils/args.py","file_url":"https://github.com/omarfoq/FedEM/blob/HEAD/utils/args.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"214218b6675ec25e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}