{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-06003","title":"EvoESAP: Non-Uniform Expert Pruning for Sparse MoE","arxiv_id":"2603.06003","date":"2026-03-06","proceeding":null,"authors":["Zongfang Liu","Shengkun Tang","Boyang Sun","Zhiqiang Shen","Xin Yuan"],"abstract":"Sparse Mixture-of-Experts (SMoE) language models achieve strong capability at low per-token compute, yet deployment remains constrained by memory footprint and throughput because the full expert pool must still be stored and served. Post-training expert pruning reduces this cost, but most methods focus on which experts to prune within each layer and default to a uniform layer-wise sparsity allocation, even though the layer-wise allocation can strongly affect performance. We decouple pruning into within-layer expert ranking and across-layer budget allocation, and introduce \\textbf{E}xpected \\textbf{S}peculative \\textbf{A}cceptance \\textbf{P}roxy (\\textbf{ESAP}), a speculative-decoding-inspired, teacher-forced metric that measures how well a pruned model matches the full model without costly autoregressive decoding. ESAP is bounded and stable, enabling cheap comparison of many candidates. Building on ESAP, we propose EvoESAP, an evolutionary search framework that finds an improved non-uniform layer-wise sparsity allocation under a fixed global budget while holding the within-layer pruning order fixed, making it a plug-and-play method for criteria such as Frequency, EAN, SEER, and REAP. Across 7B--30B SMoE LLMs at 25\\% and 50\\% sparsity, EvoESAP consistently discovers non-uniform allocations that improve open-ended generation (up to \\textbf{+19.6\\%} on MATH-500 at 50\\% sparsity) while preserving competitive multiple-choice accuracy compared with uniform pruning at the same sparsity. Code is available at https://github.com/ZongfangLiu/EvoESAP.","url_abs":"https://arxiv.org/abs/2603.06003","url_pdf":"https://arxiv.org/pdf/2603.06003","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.06003","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.06003"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ZongfangLiu/EvoESAP","reach":null}],"summary":{"ran":3,"ran_draft_wrong":7,"ran_fixture":3,"ran_violates":1,"ran_honours":2,"unverified":10},"by_repo_kind":{"found_in_text":{"samples":26,"ran":16,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a28a0961f94a3e3e","entry":"_FnHandle","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a28a0961f94a3e3e"}},{"code_sha256_prefix":"4a10c9148b426445","entry":"_SpecDecStats","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a10c9148b426445"}},{"code_sha256_prefix":"5788cceabb56c627","entry":"_SpecDecStatsTracker","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5788cceabb56c627"}},{"code_sha256_prefix":"089b7ed0672188b0","entry":"_allocate_budget","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"089b7ed0672188b0"}},{"code_sha256_prefix":"3d92470aa99eb19b","entry":"_filtered_search_args","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3d92470aa99eb19b"}},{"code_sha256_prefix":"22c4f278408b627a","entry":"_format_ratio_list","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"22c4f278408b627a"}},{"code_sha256_prefix":"ef1ec01b9465b624","entry":"_is_cuda_oom","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef1ec01b9465b624"}},{"code_sha256_prefix":"929e80ef23cd4dd1","entry":"_normalize_parities","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"929e80ef23cd4dd1"}},{"code_sha256_prefix":"d1d5d55e0be2ccee","entry":"_pattern_weights","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1d5d55e0be2ccee"}},{"code_sha256_prefix":"4038d462d1ae4fad","entry":"_plan_matches_parity","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4038d462d1ae4fad"}},{"code_sha256_prefix":"3d650e9d614d82a5","entry":"_resolve_model_max_length","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3d650e9d614d82a5"}},{"code_sha256_prefix":"d4b09afd0239c468","entry":"get_moe","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d4b09afd0239c468"}},{"code_sha256_prefix":"e78eb7cb33a62a2a","entry":"init_pattern","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e78eb7cb33a62a2a"}},{"code_sha256_prefix":"4a37d54bf9c6b01f","entry":"mutate_plan","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a37d54bf9c6b01f"}},{"code_sha256_prefix":"875ffd03124c68af","entry":"normalize_run_name_base","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"875ffd03124c68af"}},{"code_sha256_prefix":"04af135e6f03f215","entry":"plan_to_dict","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"04af135e6f03f215"}},{"code_sha256_prefix":"6718de04f32781ac","entry":"_append_jsonl_log","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6718de04f32781ac"}},{"code_sha256_prefix":"c65130df41a4ec83","entry":"_atomic_json_dump","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c65130df41a4ec83"}},{"code_sha256_prefix":"eebfc8161894a6bf","entry":"_atomic_torch_save","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eebfc8161894a6bf"}},{"code_sha256_prefix":"c07679379072607c","entry":"_cleanup_handles","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c07679379072607c"}},{"code_sha256_prefix":"c2eed2cce78288bf","entry":"_install_temp_router_masks","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c2eed2cce78288bf"}},{"code_sha256_prefix":"d43ac7e8b2b00117","entry":"_record_spec_dec_stats","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d43ac7e8b2b00117"}},{"code_sha256_prefix":"7c368d4dea1ca4cd","entry":"_spec_dec_stats_transformers","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7c368d4dea1ca4cd"}},{"code_sha256_prefix":"995a466459a11377","entry":"_validate_resume_state","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"995a466459a11377"}},{"code_sha256_prefix":"085132d4c7fe536d","entry":"evolutionary_search","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"085132d4c7fe536d"}},{"code_sha256_prefix":"d06213068931b99a","entry":"score_candidate","repo":"ZongfangLiu/EvoESAP","repo_kind":"found_in_text","path":"src/reap/search_utils.py","file_url":"https://github.com/ZongfangLiu/EvoESAP/blob/HEAD/src/reap/search_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d06213068931b99a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}