{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/communication-efficient-federated-learning-20","title":"Adaptive Compression in Federated Learning via Side Information","arxiv_id":"2306.12625","date":"2023-06-22","proceeding":null,"authors":["Berivan Isik","Francesco Pase","Deniz Gunduz","Sanmi Koyejo","Tsachy Weissman","Michele Zorzi"],"abstract":"The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL). Among existing approaches, state-of-the-art bitrate-accuracy tradeoffs have been achieved using stochastic compression methods -- in which the client $n$ sends a sample from a client-only probability distribution $q_{\\phi^{(n)}}$, and the server estimates the mean of the clients' distributions using these samples. However, such methods do not take full advantage of the FL setup where the server, throughout the training process, has side information in the form of a global distribution $p_{\\theta}$ that is close to the clients' distribution $q_{\\phi^{(n)}}$ in Kullback-Leibler (KL) divergence. In this work, we exploit this closeness between the clients' distributions $q_{\\phi^{(n)}}$'s and the side information $p_{\\theta}$ at the server, and propose a framework that requires approximately $D_{KL}(q_{\\phi^{(n)}}|| p_{\\theta})$ bits of communication. We show that our method can be integrated into many existing stochastic compression frameworks to attain the same (and often higher) test accuracy with up to $82$ times smaller bitrate than the prior work -- corresponding to 2,650 times overall compression.","url_abs":"https://arxiv.org/abs/2306.12625v3","url_pdf":"https://arxiv.org/pdf/2306.12625v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"communication-efficient-federated-learning-20","repo_url":"https://github.com/francescopase/federated-klms","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.12625","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.12625"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/francescopase/federated-klms","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b8e95809ca2c17c9","entry":"sigmoid","repo":"francescopase/federated-klms","repo_kind":"official","path":"flower_fl/clients/fedpm_client.py","file_url":"https://github.com/francescopase/federated-klms/blob/HEAD/flower_fl/clients/fedpm_client.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b8e95809ca2c17c9"}},{"code_sha256_prefix":"f2d3a731c98a9cf0","entry":"get_parameters","repo":"francescopase/federated-klms","repo_kind":"official","path":"flower_fl/utils/models.py","file_url":"https://github.com/francescopase/federated-klms/blob/HEAD/flower_fl/utils/models.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2d3a731c98a9cf0"}},{"code_sha256_prefix":"1671b368a1ab2ab0","entry":"resnet20","repo":"francescopase/federated-klms","repo_kind":"official","path":"flower_fl/models/resnet.py","file_url":"https://github.com/francescopase/federated-klms/blob/HEAD/flower_fl/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1671b368a1ab2ab0"}},{"code_sha256_prefix":"a91a0a3d5796b8d8","entry":"resnet32","repo":"francescopase/federated-klms","repo_kind":"official","path":"flower_fl/models/resnet.py","file_url":"https://github.com/francescopase/federated-klms/blob/HEAD/flower_fl/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a91a0a3d5796b8d8"}},{"code_sha256_prefix":"b37b467a20738989","entry":"resnet44","repo":"francescopase/federated-klms","repo_kind":"official","path":"flower_fl/models/resnet.py","file_url":"https://github.com/francescopase/federated-klms/blob/HEAD/flower_fl/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b37b467a20738989"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}