{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/qualitatively-characterizing-neural-network","title":"Qualitatively characterizing neural network optimization problems","arxiv_id":"1412.6544","date":"2014-12-19","proceeding":null,"authors":["Ian J. Goodfellow","Oriol Vinyals","Andrew M. Saxe"],"abstract":"Training neural networks involves solving large-scale non-convex optimization\nproblems. This task has long been believed to be extremely difficult, with fear\nof local minima and other obstacles motivating a variety of schemes to improve\noptimization, such as unsupervised pretraining. However, modern neural networks\nare able to achieve negligible training error on complex tasks, using only\ndirect training with stochastic gradient descent. We introduce a simple\nanalysis technique to look for evidence that such networks are overcoming local\noptima. We find that, in fact, on a straight path from initialization to\nsolution, a variety of state of the art neural networks never encounter any\nsignificant obstacles.","url_abs":"http://arxiv.org/abs/1412.6544v6","url_pdf":"http://arxiv.org/pdf/1412.6544v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"qualitatively-characterizing-neural-network","repo_url":"https://github.com/okn-yu/Visualizing-the-Loss-Landscape-of-Neural-Nets","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1412.6544","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}