{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-01331","title":"MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Models","arxiv_id":"2603.01331","date":"2026-03-02","proceeding":null,"authors":["Kejing Xia","Mingzhe Li","Lixuan Wei","Zhenbang Du","Xiangchi Yuan","Dachuan Shi","Qirui Jin","Wenke Lee"],"abstract":"Discrete diffusion language models (dLLMs) generate text by iteratively denoising a masked sequence. However, standard dLLMs condition each denoising step solely on the current hard-masked sequence, while intermediate continuous representations are discarded after sampling and remasking. We term this bottleneck the \\textbf{Information Island} issue: continuous information remains isolated within individual denoising steps and fails to propagate across the trajectory. This bottleneck is especially harmful for reasoning, which requires intermediate reasoning state to be preserved and updated across many denoising steps. To address this limitation, we introduce \\textbf{MetaState}, a lightweight recurrent augmentation that equips a frozen dLLM backbone with persistent, fixed-size working memory. MetaState comprises three modules with a shared time conditioner: a cross-attention \\textbf{Mixer} that reads backbone activations into memory slots, a GRU-style \\textbf{Updater} that integrates information across steps, and a cross-attention \\textbf{Injector} that writes the updated memory back into the backbone. We train these modules with a dedicated $K$-step unrolling pipeline to learn multi-step dynamics. MetaState adds only ${\\sim}0.6\\%$ trainable parameters while keeping the backbone frozen, and consistently improves reasoning performance over frozen baselines on mathematical reasoning and code generation benchmarks, with an average gain of 4.5 percentage points across all evaluations.","url_abs":"https://arxiv.org/abs/2603.01331","url_pdf":"https://arxiv.org/pdf/2603.01331","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.01331","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.01331"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Les1a/MetaState","reach":null}],"summary":{"unverified":8},"by_repo_kind":{"found_in_text":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c7fd61b6c723375b","entry":"add_gumbel_noise","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/core/samplers/utils.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/core/samplers/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c7fd61b6c723375b"}},{"code_sha256_prefix":"67f6e87d643dd456","entry":"clip_row","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/utils/data.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"67f6e87d643dd456"}},{"code_sha256_prefix":"de5487e7b83bbedd","entry":"compute_kv_replace_offset","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/core/modules/recurrent.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/core/modules/recurrent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de5487e7b83bbedd"}},{"code_sha256_prefix":"9b41aca897525bbf","entry":"create_recurrent_module_config","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/core/modules/recurrent.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/core/modules/recurrent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9b41aca897525bbf"}},{"code_sha256_prefix":"effe0c4dc9789e1b","entry":"get_num_transfer_tokens_uniform","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/core/samplers/utils.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/core/samplers/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"effe0c4dc9789e1b"}},{"code_sha256_prefix":"4c6685fbeb28fcff","entry":"get_recurrent_checkpoint_model_type","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/utils/models.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/utils/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4c6685fbeb28fcff"}},{"code_sha256_prefix":"f9026375563cfa72","entry":"is_lightweight_recurrent_checkpoint","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/utils/models.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/utils/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9026375563cfa72"}},{"code_sha256_prefix":"fb98ed14f9cf7c37","entry":"tokenize_and_group","repo":"Les1a/MetaState","repo_kind":"found_in_text","path":"dllm/utils/data.py","file_url":"https://github.com/Les1a/MetaState/blob/HEAD/dllm/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb98ed14f9cf7c37"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}