{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-global-linear-convergence-of-frank","title":"On the Global Linear Convergence of Frank-Wolfe Optimization Variants","arxiv_id":"1511.05932","date":"2015-11-18","proceeding":"NeurIPS 2015 12","authors":["Simon Lacoste-Julien","Martin Jaggi"],"abstract":"The Frank-Wolfe (FW) optimization algorithm has lately re-gained popularity\nthanks in particular to its ability to nicely handle the structured constraints\nappearing in machine learning applications. However, its convergence rate is\nknown to be slow (sublinear) when the solution lies at the boundary. A simple\nless-known fix is to add the possibility to take 'away steps' during\noptimization, an operation that importantly does not require a feasibility\noracle. In this paper, we highlight and clarify several variants of the\nFrank-Wolfe optimization algorithm that have been successfully applied in\npractice: away-steps FW, pairwise FW, fully-corrective FW and Wolfe's minimum\nnorm point algorithm, and prove for the first time that they all enjoy global\nlinear convergence, under a weaker condition than strong convexity of the\nobjective. The constant in the convergence rate has an elegant interpretation\nas the product of the (classical) condition number of the function with a novel\ngeometric quantity that plays the role of a 'condition number' of the\nconstraint set. We provide pointers to where these algorithms have made a\ndifference in practice, in particular with the flow polytope, the marginal\npolytope and the base polytope for submodular optimization.","url_abs":"http://arxiv.org/abs/1511.05932v1","url_pdf":"http://arxiv.org/pdf/1511.05932v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-global-linear-convergence-of-frank","repo_url":"https://github.com/Simon-Lacoste-Julien/linearFW","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1511.05932","atlas_url":"https://app.syntology.ai/?focus=1511.05932","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}