{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2608-18261","title":"Cacheable by Design? Training Mixture-of-Experts Routers for Locality Against the Edge Memory-Bandwidth Wall: A Pre-Registered Negative Result with a Systems Measurement Study","arxiv_id":"2608.18261","date":"2026-08-18","proceeding":null,"authors":["Shriniwas Ramesh Suram"],"abstract":"Serving a 235B-parameter Mixture-of-Experts (MoE) model on a single 8 GB GPU is bottlenecked not by compute but by memory bandwidth: decode must stream each token's active experts from whichever tier holds them, and on consumer hardware most experts sit on an SSD far slower than RAM. We quantify this bandwidth wall on Qwen3-235B (Q4_K_M, 134 GB): measured decode is 0.44 tok/s warm, matching a bytes-per-token / bandwidth model, while a batching scheme that should amortize one disk sweep instead collapses at batch 32 from paging thrash. We build llama-moe-trace, a zero-surgery router-telemetry tool, and measure routing on Qwen3-30B: adjacent-token expert reuse is 2.0x chance, 95% of traffic uses 52.5% of experts, and an LRU cache of 13.4% of experts serves 66% of requests. We then ask whether cacheability is trainable: we pre-register training of 137M MoE language models with auxiliary locality and domain router losses, under joint criteria on cache-miss reduction and perplexity. The mechanism works (misses down up to 60%; a 99% static-pin hit rate) but every configuration fails the pre-registered <=1% perplexity gate -- miss reduction and quality are tightly coupled. Concurrent StickyMoE reports the same loss as near-free on single-domain sub-25M models; on multi-domain 137M we find the tax real. Our contribution is this pre-registered, stricter-criterion, multi-domain evaluation plus edge-serving measurements. A 340M rung shows the tax does not shrink with scale (it rises slightly). We further show training-free cache-aware rerouting stacks with trained locality -- together ~80% miss reduction at <=3.4% perplexity at both sizes, far cheaper than either alone -- while domain-primed prefetching does not help. All code, traces, and the pre-registration are released.","url_abs":"https://arxiv.org/abs/2608.18261","url_pdf":"https://arxiv.org/pdf/2608.18261","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2608.18261","atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2608.18261"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Shriniwas410/cacheable-by-design","reach":null}],"summary":{"ran":13,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":15,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6abb765130ac2f62","entry":"assert_comparable","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"provenance.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/provenance.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6abb765130ac2f62"}},{"code_sha256_prefix":"4c6d5d4f5f3922c1","entry":"load","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"frontier.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/frontier.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c6d5d4f5f3922c1"}},{"code_sha256_prefix":"59599771ecfff4b3","entry":"load","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"summarize.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/summarize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59599771ecfff4b3"}},{"code_sha256_prefix":"f8447efc3a8738fd","entry":"load_prov","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"provenance.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/provenance.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8447efc3a8738fd"}},{"code_sha256_prefix":"ba8d8cfb880cdd90","entry":"load_trace","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"analyze.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/analyze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba8d8cfb880cdd90"}},{"code_sha256_prefix":"59f761c9c46dfb45","entry":"mean_metric","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"summarize.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/summarize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59f761c9c46dfb45"}},{"code_sha256_prefix":"abb634cc2829f958","entry":"mean_ppl","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"summarize.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/summarize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"abb634cc2829f958"}},{"code_sha256_prefix":"81ece1943db00ed8","entry":"miss_per_tok","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"analyze.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/analyze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81ece1943db00ed8"}},{"code_sha256_prefix":"3081d51cd278eccb","entry":"page_of","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"paper/build_ctx.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/paper/build_ctx.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3081d51cd278eccb"}},{"code_sha256_prefix":"bc0b37eed100678d","entry":"ppl","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"cert_elision.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/cert_elision.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc0b37eed100678d"}},{"code_sha256_prefix":"4e327a8a30bf2d21","entry":"reroute_token","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"cacheeval.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/cacheeval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e327a8a30bf2d21"}},{"code_sha256_prefix":"1a52e3c4ac6ee86e","entry":"reuse_lag1","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"analyze.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/analyze.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a52e3c4ac6ee86e"}},{"code_sha256_prefix":"4fbd2fe1d47ad8b0","entry":"rope_cache","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"model.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4fbd2fe1d47ad8b0"}},{"code_sha256_prefix":"07941306772c339b","entry":"apply_rope","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"model.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07941306772c339b"}},{"code_sha256_prefix":"bc9f31b489cd86a5","entry":"pts","repo":"Shriniwas410/cacheable-by-design","repo_kind":"found_in_text","path":"frontier2.py","file_url":"https://github.com/Shriniwas410/cacheable-by-design/blob/HEAD/frontier2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc9f31b489cd86a5"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9623,"papers_checked":6885},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}