{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/12024184","title":"Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences","arxiv_id":"1202.4184","date":"2012-02-19","proceeding":null,"authors":["Benjamin Recht","Christopher Re"],"abstract":"Randomized algorithms that base iteration-level decisions on samples from\nsome pool are ubiquitous in machine learning and optimization. Examples include\nstochastic gradient descent and randomized coordinate descent. This paper makes\nprogress at theoretically evaluating the difference in performance between\nsampling with- and without-replacement in such algorithms. Focusing on least\nmeans squares optimization, we formulate a noncommutative arithmetic-geometric\nmean inequality that would prove that the expected convergence rate of\nwithout-replacement sampling is faster than that of with-replacement sampling.\nWe demonstrate that this inequality holds for many classes of random matrices\nand for some pathological examples as well. We provide a deterministic\nworst-case bound on the gap between the discrepancy between the two sampling\nmodels, and explore some of the impediments to proving this inequality in full\ngenerality. We detail the consequences of this inequality for stochastic\ngradient descent and the randomized Kaczmarz algorithm for solving linear\nsystems.","url_abs":"http://arxiv.org/abs/1202.4184v1","url_pdf":"http://arxiv.org/pdf/1202.4184v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"12024184","repo_url":"https://github.com/haleyhfeng/Ordering-of-Randomized-Optimization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"12024184","repo_url":"https://github.com/leontynew/Project-2---Random-or-not-random","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"12024184","repo_url":"https://github.com/suanmingde/Ordering-of-Randomized-Optimization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1202.4184","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}