{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/nuqsgd-improved-communication-efficiency-for","title":"NUQSGD: Improved Communication Efficiency for Data-parallel SGD via Nonuniform Quantization","arxiv_id":"1908.06077","date":"2019-08-16","proceeding":null,"authors":["Ali Ramezani-Kebrya","Fartash Faghri","Daniel M. Roy"],"abstract":"As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed on clusters to perform model fitting in parallel. Alistarh et al. (2017) describe two variants of data-parallel SGD that quantize and encode gradients to lessen communication costs. For the first variant, QSGD, they provide strong theoretical guarantees. For the second variant, which we call QSGDinf, they demonstrate impressive empirical gains for distributed training of large neural networks. Building on their work, we propose an alternative scheme for quantizing gradients and show that it yields stronger theoretical guarantees than exist for QSGD while matching the empirical performance of QSGDinf.","url_abs":"https://arxiv.org/abs/1908.06077v1","url_pdf":"https://arxiv.org/pdf/1908.06077v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"nuqsgd-improved-communication-efficiency-for","repo_url":"https://github.com/fartashf/nuqsgd","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1908.06077","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1908.06077"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fartashf/nuqsgd","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"1391a74f11916fc6","entry":"imagenet_first_epoch","repo":"fartashf/nuqsgd","repo_kind":"official","path":"grid/nuq.py","file_url":"https://github.com/fartashf/nuqsgd/blob/HEAD/grid/nuq.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1391a74f11916fc6"}},{"code_sha256_prefix":"03bdaa53e1862b5f","entry":"imagenet_last_epoch","repo":"fartashf/nuqsgd","repo_kind":"official","path":"grid/nuq.py","file_url":"https://github.com/fartashf/nuqsgd/blob/HEAD/grid/nuq.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03bdaa53e1862b5f"}},{"code_sha256_prefix":"c7c4ee6b869380b1","entry":"mnist","repo":"fartashf/nuqsgd","repo_kind":"official","path":"grid/nuq.py","file_url":"https://github.com/fartashf/nuqsgd/blob/HEAD/grid/nuq.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7c4ee6b869380b1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}