{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/geometry-of-optimization-and-implicit","title":"Geometry of Optimization and Implicit Regularization in Deep Learning","arxiv_id":"1705.03071","date":"2017-05-08","proceeding":null,"authors":["Behnam Neyshabur","Ryota Tomioka","Ruslan Salakhutdinov","Nathan Srebro"],"abstract":"We argue that the optimization plays a crucial role in generalization of deep\nlearning models through implicit regularization. We do this by demonstrating\nthat generalization ability is not controlled by network size but rather by\nsome other implicit control. We then demonstrate how changing the empirical\noptimization procedure can improve generalization, even if actual optimization\nquality is not affected. We do so by studying the geometry of the parameter\nspace of deep networks, and devising an optimization algorithm attuned to this\ngeometry.","url_abs":"http://arxiv.org/abs/1705.03071v1","url_pdf":"http://arxiv.org/pdf/1705.03071v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"geometry-of-optimization-and-implicit","repo_url":"https://github.com/bneyshabur/generalization-bounds","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.03071","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}