{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/docom-sgt-doubly-compressed-momentum-assisted","title":"DoCoM: Compressed Decentralized Optimization with Near-Optimal Sample Complexity","arxiv_id":"2202.00255","date":"2022-02-01","proceeding":null,"authors":["Chung-Yiu Yau","Hoi-To Wai"],"abstract":"This paper proposes the Doubly Compressed Momentum-assisted stochastic gradient tracking algorithm $\\texttt{DoCoM}$ for communication-efficient decentralized optimization. The algorithm features two main ingredients to achieve a near-optimal sample complexity while allowing for communication compression. First, the algorithm tracks both the averaged iterate and stochastic gradient using compressed gossiping consensus. Second, a momentum step is incorporated for adaptive variance reduction with the local gradient estimates. We show that $\\texttt{DoCoM}$ finds a near-stationary solution at all participating agents satisfying $\\mathbb{E}[ \\| \\nabla f( \\theta ) \\|^2 ] = \\mathcal{O}( 1 / T^{2/3} )$ in $T$ iterations, where $f(\\theta)$ is a smooth (possibly non-convex) objective function. Notice that the proof is achieved via analytically designing a new potential function that tightly tracks the one-iteration progress of $\\texttt{DoCoM}$. As a corollary, our analysis also established the linear convergence of $\\texttt{DoCoM}$ to a global optimal solution for objective functions with the Polyak-{\\L}ojasiewicz condition. Numerical experiments demonstrate that our algorithm outperforms several state-of-the-art algorithms in practice.","url_abs":"https://arxiv.org/abs/2202.00255v2","url_pdf":"https://arxiv.org/pdf/2202.00255v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"docom-sgt-doubly-compressed-momentum-assisted","repo_url":"https://github.com/OscarYau525/docom","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2202.00255","atlas_url":"https://app.syntology.ai/?focus=2202.00255","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}