{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-loss-function-topology-with","title":"Exploring loss function topology with cyclical learning rates","arxiv_id":"1702.04283","date":"2017-02-14","proceeding":null,"authors":["Leslie N. Smith","Nicholay Topin"],"abstract":"We present observations and discussion of previously unreported phenomena\ndiscovered while training residual networks. The goal of this work is to better\nunderstand the nature of neural networks through the examination of these new\nempirical results. These behaviors were identified through the application of\nCyclical Learning Rates (CLR) and linear network interpolation. Among these\nbehaviors are counterintuitive increases and decreases in training loss and\ninstances of rapid training. For example, we demonstrate how CLR can produce\ngreater testing accuracy than traditional training despite using large learning\nrates. Files to replicate these results are available at\nhttps://github.com/lnsmith54/exploring-loss","url_abs":"http://arxiv.org/abs/1702.04283v1","url_pdf":"http://arxiv.org/pdf/1702.04283v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-loss-function-topology-with","repo_url":"https://github.com/lnsmith54/exploring-loss","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"caffe2","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"exploring-loss-function-topology-with","repo_url":"https://github.com/matheus695p/regularization-techniques-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1702.04283","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}