{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gradient-sliding-for-composite-optimization","title":"Gradient Sliding for Composite Optimization","arxiv_id":"1406.0919","date":"2014-06-04","proceeding":null,"authors":["Guanghui Lan"],"abstract":"We consider in this paper a class of composite optimization problems whose\nobjective function is given by the summation of a general smooth and nonsmooth\ncomponent, together with a relatively simple nonsmooth term. We present a new\nclass of first-order methods, namely the gradient sliding algorithms, which can\nskip the computation of the gradient for the smooth component from time to\ntime. As a consequence, these algorithms require only ${\\cal\nO}(1/\\sqrt{\\epsilon})$ gradient evaluations for the smooth component in order\nto find an $\\epsilon$-solution for the composite problem, while still\nmaintaining the optimal ${\\cal O}(1/\\epsilon^2)$ bound on the total number of\nsubgradient evaluations for the nonsmooth component. We then present a\nstochastic counterpart for these algorithms and establish similar complexity\nbounds for solving an important class of stochastic composite optimization\nproblems. Moreover, if the smooth component in the composite function is\nstrongly convex, the developed gradient sliding algorithms can significantly\nreduce the number of graduate and subgradient evaluations for the smooth and\nnonsmooth component to ${\\cal O} (\\log (1/\\epsilon))$ and ${\\cal\nO}(1/\\epsilon)$, respectively. Finally, we generalize these algorithms to the\ncase when the smooth component is replaced by a nonsmooth one possessing a\ncertain bi-linear saddle point structure.","url_abs":"http://arxiv.org/abs/1406.0919v2","url_pdf":"http://arxiv.org/pdf/1406.0919v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gradient-sliding-for-composite-optimization","repo_url":"https://github.com/Libensemble/libensemble","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1406.0919","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}