{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-07651","title":"OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference","arxiv_id":"2510.07651","date":"2025-10-09","proceeding":"ICML","authors":["Yuzhe Gu","Xiyu Liang","Jiaojiao Zhao","Enmao Diao"],"abstract":"Large language models (LLMs) with extended context windows enable powerful applications but impose significant memory overhead, as caching all key-value (KV) states scales linearly with sequence length and batch size. Existing cache eviction methods address this by exploiting attention sparsity, yet they typically rank tokens heuristically using accumulated attention weights without considering their true impact on attention outputs. We propose Optimal Brain Cache (OBCache), a principled framework that formulates cache eviction as a layer-wise structured pruning problem. Building upon the Optimal Brain Damage (OBD) theory, OBCache quantifies token saliency by measuring the perturbation in attention outputs induced by pruning tokens, with closed-form scores derived for isolated keys, isolated values, and joint key-value pairs. Our scores account not only for attention weights but also for information from value states and attention outputs, thereby enhancing existing eviction strategies with output-aware signals. Experiments on LLaMA and Qwen models demonstrate that replacing the heuristic scores in existing works, which estimate token saliency across different query positions, with OBCache's output-aware scores consistently improves long-context accuracy. Code is available at https://github.com/DreamSoul-AI/OBCache.","url_abs":"https://arxiv.org/abs/2510.07651","url_pdf":"https://arxiv.org/pdf/2510.07651","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2510.07651","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.07651"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/DreamSoul-AI/OBCache","reach":null}],"summary":{"ran_honours":1,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"775170539fb88bfc","entry":"get_kernel_launch_config","repo":"DreamSoul-AI/OBCache","repo_kind":"found_in_text","path":"kvpress/presses/obcache_press.py","file_url":"https://github.com/DreamSoul-AI/OBCache/blob/HEAD/kvpress/presses/obcache_press.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"775170539fb88bfc"}},{"code_sha256_prefix":"840c387847cebae6","entry":"OBCacheScoringMixin","repo":"DreamSoul-AI/OBCache","repo_kind":"found_in_text","path":"kvpress/presses/obcache_press.py","file_url":"https://github.com/DreamSoul-AI/OBCache/blob/HEAD/kvpress/presses/obcache_press.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"840c387847cebae6"}},{"code_sha256_prefix":"8060223b0779a88d","entry":"require_triton","repo":"DreamSoul-AI/OBCache","repo_kind":"found_in_text","path":"kvpress/presses/obcache_press.py","file_url":"https://github.com/DreamSoul-AI/OBCache/blob/HEAD/kvpress/presses/obcache_press.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8060223b0779a88d"}},{"code_sha256_prefix":"8b5a42aedf1f3a36","entry":"run_v_score_kernel","repo":"DreamSoul-AI/OBCache","repo_kind":"found_in_text","path":"kvpress/presses/obcache_press.py","file_url":"https://github.com/DreamSoul-AI/OBCache/blob/HEAD/kvpress/presses/obcache_press.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b5a42aedf1f3a36"}},{"code_sha256_prefix":"7060a9e82baa60a4","entry":"triton_is_available","repo":"DreamSoul-AI/OBCache","repo_kind":"found_in_text","path":"kvpress/presses/obcache_press.py","file_url":"https://github.com/DreamSoul-AI/OBCache/blob/HEAD/kvpress/presses/obcache_press.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7060a9e82baa60a4"}},{"code_sha256_prefix":"953df5ec1a977dbc","entry":"triton_v_score","repo":"DreamSoul-AI/OBCache","repo_kind":"found_in_text","path":"kvpress/presses/obcache_press.py","file_url":"https://github.com/DreamSoul-AI/OBCache/blob/HEAD/kvpress/presses/obcache_press.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"953df5ec1a977dbc"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}