{"url":"/method/nadam","slug":"nadam","name":"NADAM","full_name":"NADAM","full_name_withheld":false,"description_markdown":"**NADAM**, or **Nesterov-accelerated Adaptive Moment Estimation**, combines [Adam](https://paperswithcode.com/method/adam) and [Nesterov Momentum](https://paperswithcode.com/method/nesterov-accelerated-gradient). The update rule is of the form:\r\n\r\n$$ \\theta\\_{t+1} = \\theta\\_{t} - \\frac{\\eta}{\\sqrt{\\hat{v}\\_{t}}+\\epsilon}\\left(\\beta\\_{1}\\hat{m}\\_{t} + \\frac{(1-\\beta\\_{t})g\\_{t}}{1-\\beta^{t}\\_{1}}\\right)$$\r\n\r\nImage Source: [Incorporating Nesterov Momentum into Adam](http://cs229.stanford.edu/proj2015/054_report.pdf)","description_state":"present","introduced_year":2015,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":"https://github.com/Dawn-Of-Eve/nadir","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Stochastic Optimization","url":"/methods/category/stochastic-optimization","pwc_aliases":[]}],"n_papers_tagged":7,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"New Insight in Cervical Cancer Diagnosis Using Convolution Neural Network Architecture","date":"2024-10-23","arxiv_id":"2410.17735","n_code_links":0,"syntology":null},{"paper":null,"title":"Non-convergence to global minimizers in data driven supervised deep learning: Adam and stochastic gradient descent optimization provably fail to converge to global minimizers in the training of deep neural networks with ReLU activation","date":"2024-10-14","arxiv_id":"2410.10533","n_code_links":0,"syntology":null},{"paper":null,"title":"Comparison of Optimizers for Fault Isolation and Diagnostics of Control Rod Drives","date":"2024-07-09","arxiv_id":"2407.06557","n_code_links":0,"syntology":null},{"paper":null,"title":"A model for multi-attack classification to improve intrusion detection performance using deep learning approaches","date":"2023-10-25","arxiv_id":"2310.16380","n_code_links":0,"syntology":null},{"paper":"/paper/mish-a-self-regularized-non-monotonic-neural","title":"Mish: A Self Regularized Non-Monotonic Activation Function","date":"2019-08-23","arxiv_id":"1908.08681","n_code_links":9,"syntology":{"ran":3,"of":12,"unverified":9,"pointer_only":0}},{"paper":"/paper/namsg-an-efficient-method-for-training-neural","title":"An Adaptive Remote Stochastic Gradient Method for Training Neural Networks","date":"2019-05-04","arxiv_id":"1905.01422","n_code_links":1,"syntology":null},{"paper":"/paper/on-the-convergence-of-adam-and-beyond-1","title":"On the Convergence of Adam and Beyond","date":"2019-04-19","arxiv_id":"1904.09237","n_code_links":3,"syntology":{"ran":0,"of":2,"unverified":2,"pointer_only":0}}],"papers_shown":7,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":1},{"task":"/task/cancer-classification","name":"Cancer Classification","papers":1},{"task":"/task/decoder","name":"Decoder","papers":1},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/fault-detection","name":"Fault Detection","papers":1},{"task":"/task/intrusion-detection","name":"Intrusion Detection","papers":1},{"task":"/task/object-detection","name":"Object Detection","papers":1},{"task":"/task/stochastic-optimization","name":"Stochastic Optimization","papers":1},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":1},{"task":"/task/image-classification","name":"image-classification","papers":1},{"task":"/task/object-detection-1","name":"object-detection","papers":1}],"tasks_shown":12,"n_tasks":12,"usage_by_year":[{"year":"2019","papers":3},{"year":"2023","papers":1},{"year":"2024","papers":3}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/nadam"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}