{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-06884","title":"Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition","arxiv_id":"2605.06884","date":"2026-05-07","proceeding":null,"authors":["Sayantan Choudhury","Xiaoran Cheng","Martin Takáč","Sen Na","Mladen Kolar"],"abstract":"Most first-order optimizers treat matrix-valued parameters as vectors, ignoring the intrinsic geometry of hidden-layer weights in neural networks. Muon addresses this mismatch by updating along the polar factor of a momentum matrix, but its theoretical understanding has lagged behind practice. In particular, practical implementations incorporate Nesterov momentum, compute the polar factor only approximately, and operate with stochastic gradients that may be heavy-tailed. We close this gap by developing a convergence theory for Muon with Nesterov momentum and inexact polar decomposition in non-convex matrix optimization under heavy-tailed noise. Our analysis builds on a unified framework for inexact polar decomposition that captures practical iterative approximations such as Newton-Schulz and quantifies how their errors propagate through the optimization dynamics. Under this framework, we establish an optimal iteration and sample complexity of $O \\left(\\varepsilon^{\\frac{-(3α-2)}{(α-1)}} \\right)$ for finding an $\\varepsilon$-stationary point, where $α\\in(1,2]$ denotes the heavy-tail index. For the inexact-polar setting with $σ_1=0$, we also provide guarantees that do not require prior knowledge of $α$. We analyze a randomized low-rank polar decomposition that is substantially more efficient than full-space methods while remaining compatible with our theory. Numerical experiments further demonstrate the effectiveness of the proposed inexact and randomized variants.","url_abs":"https://arxiv.org/abs/2605.06884","url_pdf":"https://arxiv.org/pdf/2605.06884","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.06884","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.06884"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd","reach":null}],"summary":{"ran":9,"unverified":9},"by_repo_kind":{"found_in_text":{"samples":18,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":18,"samples":[{"code_sha256_prefix":"ec520ee882f0b378","entry":"batch_crop","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/legacy/airbench94.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/legacy/airbench94.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec520ee882f0b378"}},{"code_sha256_prefix":"edd47cb681681aaa","entry":"batch_flip_lr","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/utils.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"edd47cb681681aaa"}},{"code_sha256_prefix":"5d81a95405659114","entry":"evaluate","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/utils.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d81a95405659114"}},{"code_sha256_prefix":"68dce0deb8b45a56","entry":"get_patches","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/legacy/airbench94.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/legacy/airbench94.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"68dce0deb8b45a56"}},{"code_sha256_prefix":"3e1420c785b4714e","entry":"infer","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/utils.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e1420c785b4714e"}},{"code_sha256_prefix":"35f2b8124a9eeb69","entry":"make_net93","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench93.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench93.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35f2b8124a9eeb69"}},{"code_sha256_prefix":"eb4e45e96f4ac5ca","entry":"make_net94","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench94.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench94.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb4e45e96f4ac5ca"}},{"code_sha256_prefix":"082d51b724c5b0da","entry":"make_net95","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench95.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench95.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"082d51b724c5b0da"}},{"code_sha256_prefix":"1e472c5f6e5419da","entry":"round_up_to_multiple","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"nanogpt/train_gpt.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/nanogpt/train_gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1e472c5f6e5419da"}},{"code_sha256_prefix":"a925da5a80f8e04e","entry":"cubic_ns_theoretical","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench94_muon.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench94_muon.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a925da5a80f8e04e"}},{"code_sha256_prefix":"e4294370eb7980a3","entry":"mm_t_op","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"nanogpt/train_gpt.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/nanogpt/train_gpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e4294370eb7980a3"}},{"code_sha256_prefix":"eb4b6088ac190a92","entry":"quintic_ns_empirical","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench94_muon.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench94_muon.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb4b6088ac190a92"}},{"code_sha256_prefix":"52e11ffe09043998","entry":"quintic_ns_theoretical","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench94_muon.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench94_muon.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52e11ffe09043998"}},{"code_sha256_prefix":"007184ac0fadb5a4","entry":"train93","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench93.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench93.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"007184ac0fadb5a4"}},{"code_sha256_prefix":"76206a2719d48904","entry":"train94","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench94.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench94.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76206a2719d48904"}},{"code_sha256_prefix":"6c193743c3aae9e6","entry":"train95","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench95.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench95.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c193743c3aae9e6"}},{"code_sha256_prefix":"fe957266968b7dd2","entry":"train96","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/airbench/lib_airbench96.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/airbench/lib_airbench96.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe957266968b7dd2"}},{"code_sha256_prefix":"f3ac1549c0a496f4","entry":"zeropower_via_newtonschulz5","repo":"Xiaoran-Cheng/muon-randomized-svd","repo_kind":"found_in_text","path":"cifar10/research/airbench94_muon_simple.py","file_url":"https://github.com/Xiaoran-Cheng/muon-randomized-svd/blob/HEAD/cifar10/research/airbench94_muon_simple.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f3ac1549c0a496f4"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}