{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tracking-the-gradients-using-the-hessian-a","title":"Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods","arxiv_id":"1710.07462","date":"2017-10-20","proceeding":null,"authors":["Robert M. Gower","Nicolas Le Roux","Francis Bach"],"abstract":"Our goal is to improve variance reducing stochastic methods through better\ncontrol variates. We first propose a modification of SVRG which uses the\nHessian to track gradients over time, rather than to recondition, increasing\nthe correlation of the control variates and leading to faster theoretical\nconvergence close to the optimum. We then propose accurate and computationally\nefficient approximations to the Hessian, both using a diagonal and a low-rank\nmatrix. Finally, we demonstrate the effectiveness of our method on a wide range\nof problems.","url_abs":"http://arxiv.org/abs/1710.07462v3","url_pdf":"http://arxiv.org/pdf/1710.07462v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tracking-the-gradients-using-the-hessian-a","repo_url":"https://github.com/gowerrobert/StochOpt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1710.07462","atlas_url":"https://app.syntology.ai/?focus=1710.07462","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}