{"url":"/method/aggmo","slug":"aggmo","name":"AggMo","full_name":"AggMo","full_name_withheld":false,"description_markdown":"**Aggregated Momentum (AggMo)** is a variant of the [classical momentum](https://paperswithcode.com/method/sgd-with-momentum) stochastic optimizer which maintains several velocity vectors with different $\\beta$ parameters. AggMo averages the velocity vectors when updating the parameters. It resolves the problem of choosing a momentum parameter by taking a linear combination of multiple momentum buffers. Each of $K$ momentum buffers have a different discount factor $\\beta \\in \\mathbb{R}^{K}$, and these are averaged for the update. The update rule is:\r\n\r\n$$ \\textbf{v}\\_{t}^{\\left(i\\right)} = \\beta^{(i)}\\textbf{v}\\_{t-1}^{\\left(i\\right)} - \\nabla\\_{\\theta}f\\left(\\mathbf{\\theta}\\_{t-1}\\right) $$\r\n\r\n$$ \\mathbf{\\theta\\_{t}} = \\mathbf{\\theta\\_{t-1}} + \\frac{\\gamma\\_{t}}{K}\\sum^{K}\\_{i=1}\\textbf{v}\\_{t}^{\\left(i\\right)} $$\r\n\r\nwhere $v^{\\left(i\\right)}_{0}$ for each $i$. The vector $\\mathcal{\\beta} = \\left[\\beta^{(1)}, \\ldots, \\beta^{(K)}\\right]$ is the dampening factor.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Aggregated Momentum: Stability Through Passive Damping","paper":"/paper/aggregated-momentum-stability-through-passive","first_author":"James Lucas","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/aggregated-momentum-stability-through-passive"},"source":{"url":"http://arxiv.org/abs/1804.00325v3","title":"Aggregated Momentum: Stability Through Passive Damping","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/AtheMathmo/AggMo/blob/f91730d1c3d8398abb19ab3331e08d39cd5df390/aggmo.py#L5","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Stochastic Optimization","url":"/methods/category/stochastic-optimization","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/aggregated-momentum-stability-through-passive","title":"Aggregated Momentum: Stability Through Passive Damping","date":"2018-04-01","arxiv_id":"1804.00325","n_code_links":1,"syntology":null}],"papers_shown":1,"tasks":[],"tasks_shown":0,"n_tasks":0,"usage_by_year":[{"year":"2018","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/aggmo"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}