{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-22994","title":"A Physical Response-and-Memory Model for Muon Optimization","arxiv_id":"2608.22994","date":"2026-08-24","proceeding":null,"authors":["Yinze Hu","Hongjun Xiang","Xingao Gong","Hongyu Yu"],"abstract":"Training large language models is costly. How low a loss the same compute can ultimately reach depends on how each step's gradient is converted into a weight update; the rule that performs this conversion is the optimizer. From SGD and AdamW to the recent Muon, effective update rules have mostly been shaped by engineering intuition and then selected on benchmarks. Muon semi-orthogonalizes the momentum matrix before applying the update and has kept breaking records on public training benchmarks; yet why the semi-orthogonalized direction works, and over how long a history the momentum should average, are two questions at present answered mainly by experience. Here we treat the weight matrix during training as a responsive medium with memory and build a physical model for it, in which both questions find answers: the semi-orthogonalized direction is the maximally dissipative response under an output-side safety budget, which explains why it works; momentum is the internal stress accumulated by the medium; how long it should average is set by the relaxation of this stress, and a real medium relaxes on more than one timescale, the simplest form being one fast and one slow. On this basis we propose the Bi-Maxwell optimizer. The framework further yields a testable consequence: gradient directions change fast early in training and more slowly later, so the optimal memory length should grow with training stage; step-by-step measurements of a proxy for it by a read-only probe across 8 independent training trajectories are consistent with this consequence. Replacing the memory kernel alone, from a single timescale to two, brings training to the target loss in noticeably fewer steps on a public large-language-model optimizer benchmark.","url_abs":"https://arxiv.org/abs/2608.22994","url_pdf":"https://arxiv.org/pdf/2608.22994","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2608.22994","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.22994"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/orange4664/bimaxwell-track3-reproduction","reach":null}],"summary":{"ran_honours":2,"ran_fixture":1,"ran_draft_wrong":10,"ran_violates":3,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":20,"ran":16,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":20,"samples":[{"code_sha256_prefix":"4c5caaeadd512d58","entry":"_ns_inner","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c5caaeadd512d58"}},{"code_sha256_prefix":"f29235831eec7dd1","entry":"bounded_trust_gate","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f29235831eec7dd1"}},{"code_sha256_prefix":"730a72cc97f0361d","entry":"early_trust_floor_for_step","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"730a72cc97f0361d"}},{"code_sha256_prefix":"77e9519a772625de","entry":"gram_frobenius_norm_estimate","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77e9519a772625de"}},{"code_sha256_prefix":"183b10ffbfa21ec3","entry":"is_attn_param","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"183b10ffbfa21ec3"}},{"code_sha256_prefix":"7b9d5f143ab16957","entry":"is_attn_proj_param","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b9d5f143ab16957"}},{"code_sha256_prefix":"c6b79e26c521b25a","entry":"muon_update","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c6b79e26c521b25a"}},{"code_sha256_prefix":"477f88f3d4613108","entry":"scale_radial_update","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"477f88f3d4613108"}},{"code_sha256_prefix":"61ebeac6b99f1e86","entry":"should_soap_param","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61ebeac6b99f1e86"}},{"code_sha256_prefix":"216f11965e436420","entry":"soap_basis_qr","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"216f11965e436420"}},{"code_sha256_prefix":"53667017b85dbc3a","entry":"soap_eigenbasis","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"53667017b85dbc3a"}},{"code_sha256_prefix":"a22ce6e6b9dfc9eb","entry":"soap_precondition_momentum","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a22ce6e6b9dfc9eb"}},{"code_sha256_prefix":"6ac07cf43231f4c6","entry":"target_radius_after_update","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ac07cf43231f4c6"}},{"code_sha256_prefix":"f39fe7cee6911d97","entry":"tensor_cosine","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f39fe7cee6911d97"}},{"code_sha256_prefix":"32543a283b95aa67","entry":"trust_gate","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32543a283b95aa67"}},{"code_sha256_prefix":"c9dd31c1e98b4ebf","entry":"zeropower_via_newtonschulz5","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9dd31c1e98b4ebf"}},{"code_sha256_prefix":"b0ccd1aba8c26d17","entry":"Muon","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0ccd1aba8c26d17"}},{"code_sha256_prefix":"8e5f7c6e37c5ad58","entry":"norm_preserving_blend","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e5f7c6e37c5ad58"}},{"code_sha256_prefix":"5a77b27aefe42108","entry":"rescale_to_radius","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a77b27aefe42108"}},{"code_sha256_prefix":"eb477477ea927265","entry":"soap_update_preconditioner","repo":"orange4664/bimaxwell-track3-reproduction","repo_kind":"found_in_text","path":"record_2635_submission/train_gpt_bimaxwell_2635.py","file_url":"https://github.com/orange4664/bimaxwell-track3-reproduction/blob/HEAD/record_2635_submission/train_gpt_bimaxwell_2635.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb477477ea927265"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}