{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-01622","title":"Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models","arxiv_id":"2604.01622","date":"2026-04-02","proceeding":null,"authors":["Shuibai Zhang","Caspian Zhuang","Chihan Cui","Zhihan Yang","Fred Zhangzhi Peng","Yanxin Zhang","Haoyue Bai","Zack Jia","Yang Zhou","Guanhua Chen","Ming Liu"],"abstract":"Diffusion language models (DLMs) enable parallel, non-autoregressive text generation, yet existing DLM mixture-of-experts (MoE) models inherit token-choice (TC) routing from autoregressive systems, leading to load imbalance and rigid computation allocation. We show that expert-choice (EC) routing is a better fit for DLMs: it provides deterministic load balancing by design, yielding higher throughput and faster convergence than TC. Building on the property that EC capacity is externally controllable, we introduce timestep-dependent expert capacity, which varies expert allocation according to the denoising step. We find that allocating more capacity to low-mask-ratio steps consistently achieves the best performance under matched FLOPs, and provide a mechanistic explanation: tokens in low-mask-ratio contexts exhibit an order-of-magnitude higher learning efficiency, so concentrating compute on these steps yields the largest marginal return. Finally, we show that existing pretrained TC DLMs can be retrofitted to EC by replacing only the router, achieving faster convergence and improved accuracy across diverse downstream tasks. Together, these results establish EC routing as a superior paradigm for DLM MoE models and demonstrate that computation in DLMs can be treated as an adaptive policy rather than a fixed architectural constant. Code is available at https://github.com/zhangshuibai/EC-DLM.","url_abs":"https://arxiv.org/abs/2604.01622","url_pdf":"https://arxiv.org/pdf/2604.01622","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2604.01622","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.01622"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/zhangshuibai/EC-DLM","reach":null}],"summary":{"ran":3,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"a8e168d1e5b05b98","entry":"get_config_logger_path","repo":"zhangshuibai/EC-DLM","repo_kind":"found_in_text","path":"megatron/core/config_logger.py","file_url":"https://github.com/zhangshuibai/EC-DLM/blob/HEAD/megatron/core/config_logger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8e168d1e5b05b98"}},{"code_sha256_prefix":"36369126d19a4d23","entry":"get_path_count","repo":"zhangshuibai/EC-DLM","repo_kind":"found_in_text","path":"megatron/core/config_logger.py","file_url":"https://github.com/zhangshuibai/EC-DLM/blob/HEAD/megatron/core/config_logger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36369126d19a4d23"}},{"code_sha256_prefix":"72674f098527fbb5","entry":"has_config_logger_enabled","repo":"zhangshuibai/EC-DLM","repo_kind":"found_in_text","path":"megatron/core/config_logger.py","file_url":"https://github.com/zhangshuibai/EC-DLM/blob/HEAD/megatron/core/config_logger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72674f098527fbb5"}},{"code_sha256_prefix":"ca8b49d153c6f667","entry":"extra_args_provider","repo":"zhangshuibai/EC-DLM","repo_kind":"found_in_text","path":"custom_args/difflm.py","file_url":"https://github.com/zhangshuibai/EC-DLM/blob/HEAD/custom_args/difflm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ca8b49d153c6f667"}},{"code_sha256_prefix":"59a142f8f6eaf7b8","entry":"get_base_model","repo":"zhangshuibai/EC-DLM","repo_kind":"found_in_text","path":"pretrain_difflm.py","file_url":"https://github.com/zhangshuibai/EC-DLM/blob/HEAD/pretrain_difflm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"59a142f8f6eaf7b8"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}