{"url":"/method/polyak-averaging","slug":"polyak-averaging","name":"Polyak Averaging","full_name":"Polyak Averaging","full_name_withheld":false,"description_markdown":"**Polyak Averaging** is an optimization technique that sets final parameters to an average of (recent) parameters visited in the optimization trajectory. Specifically if in $t$ iterations we have parameters $\\theta\\_{1}, \\theta\\_{2}, \\dots, \\theta\\_{t}$, then Polyak Averaging suggests setting \r\n\r\n$$ \\theta\\_t =\\frac{1}{t}\\sum\\_{i}\\theta\\_{i} $$\r\n\r\nImage Credit: [Shubhendu Trivedi & Risi Kondor](https://ttic.uchicago.edu/~shubhendu/Pages/Files/Lecture6_flat.pdf)","description_state":"present","introduced_year":1991,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Stochastic Optimization","url":"/methods/category/stochastic-optimization","pwc_aliases":[]}],"n_papers_tagged":3,"archive_num_papers":3,"papers_newest_first":[{"paper":"/paper/statistical-inference-for-online-algorithms","title":"Statistical Inference for Online Algorithms","date":"2025-05-22","arxiv_id":"2505.17300","n_code_links":1,"syntology":null},{"paper":"/paper/fast-neural-architecture-search-of-compact","title":"Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells","date":"2018-10-25","arxiv_id":"1810.10804","n_code_links":4,"syntology":null},{"paper":"/paper/going-deeper-with-convolutions","title":"Going Deeper with Convolutions","date":"2014-09-17","arxiv_id":"1409.4842","n_code_links":83,"syntology":{"ran":27,"of":42,"unverified":15,"pointer_only":20}}],"papers_shown":3,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/depth-estimation","name":"Depth Estimation","papers":1},{"task":"/task/depth-prediction","name":"Depth Prediction","papers":1},{"task":null,"name":"GPU","papers":1},{"task":"/task/classification","name":"General Classification","papers":1},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":1},{"task":"/task/knowledge-distillation","name":"Knowledge Distillation","papers":1},{"task":"/task/language-modelling","name":"Language Modelling","papers":1},{"task":"/task/monocular-depth-estimation","name":"Monocular Depth Estimation","papers":1},{"task":"/task/architecture-search","name":"Neural Architecture Search","papers":1},{"task":"/task/object-detection","name":"Object Detection","papers":1},{"task":"/task/object-recognition","name":"Object Recognition","papers":1},{"task":"/task/pose-estimation","name":"Pose Estimation","papers":1},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":1},{"task":"/task/segmentation","name":"Segmentation","papers":1},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":1},{"task":"/task/image-classification","name":"image-classification","papers":1},{"task":null,"name":"valid","papers":1}],"tasks_shown":18,"n_tasks":18,"usage_by_year":[{"year":"2014","papers":1},{"year":"2018","papers":1},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/polyak-averaging"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}