{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variance-reduction-is-an-antidote-to","title":"Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top","arxiv_id":"2206.00529","date":"2022-06-01","proceeding":null,"authors":["Eduard Gorbunov","Samuel Horváth","Peter Richtárik","Gauthier Gidel"],"abstract":"Byzantine-robustness has been gaining a lot of attention due to the growth of the interest in collaborative and federated learning. However, many fruitful directions, such as the usage of variance reduction for achieving robustness and communication compression for reducing communication costs, remain weakly explored in the field. This work addresses this gap and proposes Byz-VR-MARINA - a new Byzantine-tolerant method with variance reduction and compression. A key message of our paper is that variance reduction is key to fighting Byzantine workers more effectively. At the same time, communication compression is a bonus that makes the process more communication efficient. We derive theoretical convergence guarantees for Byz-VR-MARINA outperforming previous state-of-the-art for general non-convex and Polyak-Lojasiewicz loss functions. Unlike the concurrent Byzantine-robust methods with variance reduction and/or compression, our complexity results are tight and do not rely on restrictive assumptions such as boundedness of the gradients or limited compression. Moreover, we provide the first analysis of a Byzantine-tolerant method supporting non-uniform sampling of stochastic gradients. Numerical experiments corroborate our theoretical findings.","url_abs":"https://arxiv.org/abs/2206.00529v3","url_pdf":"https://arxiv.org/pdf/2206.00529v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variance-reduction-is-an-antidote-to","repo_url":"https://github.com/samuelhorvath/vr_byzantine","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"federated-learning","task_name":"Federated Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2206.00529","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.00529"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/epfml/byzantine-robust-noniid-optimizer","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/SamuelHorvath/VR_Byzantine","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/samuelhorvath/vr_byzantine","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1},"found_in_text":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"250000763496ff87","entry":"Compressor","repo":"samuelhorvath/vr_byzantine","repo_kind":"official","path":"code/worker.py","file_url":"https://github.com/samuelhorvath/vr_byzantine/blob/HEAD/code/worker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"250000763496ff87"}},{"code_sha256_prefix":"1b9d3a564242e743","entry":"Identity","repo":"samuelhorvath/vr_byzantine","repo_kind":"official","path":"code/worker.py","file_url":"https://github.com/samuelhorvath/vr_byzantine/blob/HEAD/code/worker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1b9d3a564242e743"}},{"code_sha256_prefix":"75058ceb65d7ad26","entry":"MarinaWorker","repo":"samuelhorvath/vr_byzantine","repo_kind":"official","path":"code/worker.py","file_url":"https://github.com/samuelhorvath/vr_byzantine/blob/HEAD/code/worker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75058ceb65d7ad26"}},{"code_sha256_prefix":"f89c470db67c1e33","entry":"RandomNumber","repo":"samuelhorvath/vr_byzantine","repo_kind":"official","path":"code/worker.py","file_url":"https://github.com/samuelhorvath/vr_byzantine/blob/HEAD/code/worker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f89c470db67c1e33"}},{"code_sha256_prefix":"6756da01cad8bd68","entry":"TorchWorker","repo":"samuelhorvath/vr_byzantine","repo_kind":"official","path":"code/worker.py","file_url":"https://github.com/samuelhorvath/vr_byzantine/blob/HEAD/code/worker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6756da01cad8bd68"}},{"code_sha256_prefix":"87318dec6ef6b4d3","entry":"_BaseAggregator","repo":"epfml/byzantine-robust-noniid-optimizer","repo_kind":"found_in_text","path":"codes/aggregator/safeguard.py","file_url":"https://github.com/epfml/byzantine-robust-noniid-optimizer/blob/HEAD/codes/aggregator/safeguard.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87318dec6ef6b4d3"}},{"code_sha256_prefix":"7fb586f6533a2620","entry":"Safeguard","repo":"epfml/byzantine-robust-noniid-optimizer","repo_kind":"found_in_text","path":"codes/aggregator/safeguard.py","file_url":"https://github.com/epfml/byzantine-robust-noniid-optimizer/blob/HEAD/codes/aggregator/safeguard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fb586f6533a2620"}},{"code_sha256_prefix":"f96e16a75b8645c5","entry":"log","repo":"epfml/byzantine-robust-noniid-optimizer","repo_kind":"found_in_text","path":"codes/aggregator/safeguard.py","file_url":"https://github.com/epfml/byzantine-robust-noniid-optimizer/blob/HEAD/codes/aggregator/safeguard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f96e16a75b8645c5"}},{"code_sha256_prefix":"82443c01d733f55e","entry":"log_dict","repo":"epfml/byzantine-robust-noniid-optimizer","repo_kind":"found_in_text","path":"codes/aggregator/safeguard.py","file_url":"https://github.com/epfml/byzantine-robust-noniid-optimizer/blob/HEAD/codes/aggregator/safeguard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82443c01d733f55e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}