{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-distributed-quasi-newton-algorithm-for","title":"A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization","arxiv_id":"1803.01370","date":"2018-03-04","proceeding":null,"authors":["Ching-pei Lee","Cong Han Lim","Stephen J. Wright"],"abstract":"We propose a communication- and computation-efficient distributed\noptimization algorithm using second-order information for solving ERM problems\nwith a nonsmooth regularization term. Current second-order and quasi-Newton\nmethods for this problem either do not work well in the distributed setting or\nwork only for specific regularizers. Our algorithm uses successive quadratic\napproximations, and we describe how to maintain an approximation of the Hessian\nand solve subproblems efficiently in a distributed manner. The proposed method\nenjoys global linear convergence for a broad range of non-strongly convex\nproblems that includes the most commonly used ERMs, thus requiring lower\ncommunication complexity. It also converges on non-convex problems, so has the\npotential to be used on applications such as deep learning. Initial\ncomputational results on convex problems demonstrate that our method\nsignificantly improves on communication cost and running time over the current\nstate-of-the-art methods.","url_abs":"http://arxiv.org/abs/1803.01370v2","url_pdf":"http://arxiv.org/pdf/1803.01370v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-distributed-quasi-newton-algorithm-for","repo_url":"https://github.com/leepei/dplbfgs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"distributed-optimization","task_name":"Distributed Optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}