{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/eve-a-gradient-based-optimization-method-with","title":"Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates","arxiv_id":"1611.01505","date":"2016-11-04","proceeding":null,"authors":["Hiroaki Hayashi","Jayanth Koushik","Graham Neubig"],"abstract":"Adaptive gradient methods for stochastic optimization adjust the learning\nrate for each parameter locally. However, there is also a global learning rate\nwhich must be tuned in order to get the best performance. In this paper, we\npresent a new algorithm that adapts the learning rate locally for each\nparameter separately, and also globally for all parameters together.\nSpecifically, we modify Adam, a popular method for training deep learning\nmodels, with a coefficient that captures properties of the objective function.\nEmpirically, we show that our method, which we call Eve, outperforms Adam and\nother popular methods in training deep neural networks, like convolutional\nneural networks for image classification, and recurrent neural networks for\nlanguage tasks.","url_abs":"http://arxiv.org/abs/1611.01505v3","url_pdf":"http://arxiv.org/pdf/1611.01505v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"eve-a-gradient-based-optimization-method-with","repo_url":"https://github.com/rooa/eve","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"eve-a-gradient-based-optimization-method-with","repo_url":"https://github.com/AlexandruBurlacu/keras_squeezenet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"eve-a-gradient-based-optimization-method-with","repo_url":"https://github.com/K2OTO/Eve","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"eve-a-gradient-based-optimization-method-with","repo_url":"https://github.com/moskomule/eve.pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"eve-a-gradient-based-optimization-method-with","repo_url":"https://github.com/muupan/chainer-eve","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"adam","method_name":"Adam"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}