{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-06763","title":"Sparse Attention as a Range Searching Problem: Towards an Inference-Efficient Index for KV Cache","arxiv_id":"2605.06763","date":"2026-05-07","proceeding":null,"authors":["Mohsen Dehghankar","Abolfazl Asudeh"],"abstract":"Sparse attention improves LLM inference efficiency by selecting a subset of key-value entries, but at the cost of potential accuracy degradation. In particular, omitting critical KV entries can induce substantial errors in model outputs. Existing methods typically operate under fixed or adaptive token budgets and provide empirical robustness or partial theoretical guarantees, yet they do not ensure zero false negatives in decoding steps, particularly since the set of relevant tokens is both query- and step-dependent. Our empirical observations confirm that missing even one critical key can lead to sharp error spikes, especially in long reasoning tasks where the set of important tokens varies throughout decoding. This observation motivates the need for indexing methods that dynamically adapt to these variations across decoding steps while guaranteeing a full recall of the relevant keys above a certain threshold. We address this challenge by reformulating sparse attention as the halfspace range searching problem. However, existing range searching indices are not suitable for modern LLM inference due to their computational and implementation overheads. To overcome this, we introduce Louver, a novel index structure tailored for efficient KV cache retrieval. Louver (i) guarantees zero false negatives with respect to a specified threshold in both theory and practice, (ii) is lightweight to integrate into existing LLM pipelines, and (iii) incorporates hardware-aware optimizations for both CPU and GPU executions. Our experiments demonstrate that Louver outperforms prior sparse attention methods in both accuracy and runtime, and is faster than highly optimized dense attentions such as FlashAttention. These results highlight that recall guarantees are a critical and overlooked dimension of sparse attention, and open a new direction for building theoretically grounded, efficient KV cache indices.","url_abs":"https://arxiv.org/abs/2605.06763","url_pdf":"https://arxiv.org/pdf/2605.06763","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.06763"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/UIC-InDeXLab/Louver","reach":null}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"dfe2e7285eac9ae0","entry":"cluster_kmeans","repo":"UIC-InDeXLab/Louver","repo_kind":"found_in_text","path":"benchmark_area/pruning_v2/method_comparison_bench.py","file_url":"https://github.com/UIC-InDeXLab/Louver/blob/HEAD/benchmark_area/pruning_v2/method_comparison_bench.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dfe2e7285eac9ae0"}},{"code_sha256_prefix":"f9fc3f6ae6e01591","entry":"cluster_pca_kmeans","repo":"UIC-InDeXLab/Louver","repo_kind":"found_in_text","path":"benchmark_area/pruning_v2/method_config_sweep_bench.py","file_url":"https://github.com/UIC-InDeXLab/Louver/blob/HEAD/benchmark_area/pruning_v2/method_config_sweep_bench.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9fc3f6ae6e01591"}},{"code_sha256_prefix":"aa52cde564230ac9","entry":"cluster_random_projection","repo":"UIC-InDeXLab/Louver","repo_kind":"found_in_text","path":"benchmark_area/pruning_v2/method_comparison_bench.py","file_url":"https://github.com/UIC-InDeXLab/Louver/blob/HEAD/benchmark_area/pruning_v2/method_comparison_bench.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa52cde564230ac9"}},{"code_sha256_prefix":"685daa5c98f38964","entry":"cluster_spherical_kmeans","repo":"UIC-InDeXLab/Louver","repo_kind":"found_in_text","path":"benchmark_area/pruning_v2/method_comparison_bench.py","file_url":"https://github.com/UIC-InDeXLab/Louver/blob/HEAD/benchmark_area/pruning_v2/method_comparison_bench.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"685daa5c98f38964"}},{"code_sha256_prefix":"acf9d5f374ebcd9c","entry":"sdp_attention_ref","repo":"UIC-InDeXLab/Louver","repo_kind":"found_in_text","path":"attention/baseline_attn.py","file_url":"https://github.com/UIC-InDeXLab/Louver/blob/HEAD/attention/baseline_attn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"acf9d5f374ebcd9c"}},{"code_sha256_prefix":"baf9570907523ceb","entry":"cluster_pq_subspace_faiss","repo":"UIC-InDeXLab/Louver","repo_kind":"found_in_text","path":"benchmark_area/pruning_v2/method_config_sweep_bench.py","file_url":"https://github.com/UIC-InDeXLab/Louver/blob/HEAD/benchmark_area/pruning_v2/method_config_sweep_bench.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"baf9570907523ceb"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}