{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-19660","title":"OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond","arxiv_id":"2605.19660","date":"2026-05-19","proceeding":null,"authors":["Zunhai Su","Rui Yang","Chao Zhang","Yaxiu Liu","Yifan Zhang","Wei Wu","Jing Xiong","Dayou Du","Xialie Zhuang","Yulei Qian","Yuchen Xie","Yik-Chung Wu","Hongxia Yang","Ngai Wong"],"abstract":"The rapid advancement toward long-context reasoning and multi-modal intelligence has made the memory footprint of the Key-Value (KV) cache a dominant memory bottleneck for efficient deployment. While the established per-channel quantization effectively accommodates intrinsic channel-wise outliers in Key tensors, its efficacy diminishes under extreme compression. In this work, we revisit the inherent limitations of the per-channel quantization paradigm from both empirical and theoretical perspectives. Our analysis identifies Token Norm Imbalance (TNI) as the primary bottleneck to quantization fidelity. We demonstrate that TNI systematically amplifies errors when shared quantization parameters are required to span token groups exhibiting substantial norm disparities. Instead of relying on intricate quantization pipelines (e.g., TurboQuant), we propose OScaR (Omni-Scaled Canalized Rotation), an accurate and lightweight KV cache compression framework for X-LLMs (i.e., text-only, multi-modal, and omni-modal LLMs). Advancing the per-channel paradigm, OScaR employs Canalized Rotation followed by Omni-Token Scaling to mitigate TNI-induced sequence-dimensional variance both effectively and efficiently, further supported by our optimized system design and CUDA kernels. Extensive evaluations across X-LLMs show that OScaR consistently outperforms existing methods and achieves near-lossless performance under INT2 quantization, establishing it as a robust, low-complexity, and universal framework that defines a new Pareto front. Compared with the BF16 FlashDecoding-v2 baseline, our OScaR implementation achieves a notable up to 3.0x speedup in decoding, reduces memory footprint by 5.3x, and increases throughput by 4.1x. The code for OScaR is publicly available at https://github.com/ZunhaiSu/OScaR-KV-Quant.","url_abs":"https://arxiv.org/abs/2605.19660","url_pdf":"https://arxiv.org/pdf/2605.19660","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.19660","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.19660"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ZunhaiSu/OScaR-KV-Quant","reach":null}],"summary":{"ran":1,"ran_draft_wrong":5,"ran_violates":3,"ran_fixture":1,"ran_honours":3,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":15,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1ffea55c1a42d726","entry":"KVQuantizer","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ffea55c1a42d726"}},{"code_sha256_prefix":"35333e487bbf174b","entry":"_hadamard_transform_pytorch","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35333e487bbf174b"}},{"code_sha256_prefix":"562651e901c23867","entry":"_is_pow2","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"562651e901c23867"}},{"code_sha256_prefix":"9b414b96d293cc58","entry":"_norm_factor_extract","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b414b96d293cc58"}},{"code_sha256_prefix":"237295470464b4d1","entry":"_norm_factor_restore","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"237295470464b4d1"}},{"code_sha256_prefix":"42da3755ec5f55fd","entry":"asym_dequant","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42da3755ec5f55fd"}},{"code_sha256_prefix":"61f9d2c394b964e5","entry":"asym_quant","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61f9d2c394b964e5"}},{"code_sha256_prefix":"ee1603b6a317f6ef","entry":"asym_quant_dequant","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ee1603b6a317f6ef"}},{"code_sha256_prefix":"c1d9e09a87dcf2ca","entry":"get_minq_maxq","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1d9e09a87dcf2ca"}},{"code_sha256_prefix":"13baf68cd36c7523","entry":"is_pow2","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13baf68cd36c7523"}},{"code_sha256_prefix":"be82a2289d043887","entry":"sym_dequant","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be82a2289d043887"}},{"code_sha256_prefix":"03c46353b2aef040","entry":"sym_quant","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03c46353b2aef040"}},{"code_sha256_prefix":"9fef7c2350080337","entry":"sym_quant_dequant","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fef7c2350080337"}},{"code_sha256_prefix":"e782a01051c60864","entry":"QuaRotKVCacheQuantizer","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e782a01051c60864"}},{"code_sha256_prefix":"50add3266529eabf","entry":"hadamard_rotation","repo":"ZunhaiSu/OScaR-KV-Quant","repo_kind":"found_in_text","path":"kv_cache_compression/quarot_utils.py","file_url":"https://github.com/ZunhaiSu/OScaR-KV-Quant/blob/HEAD/kv_cache_compression/quarot_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50add3266529eabf"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}