{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/asvd-activation-aware-singular-value","title":"ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models","arxiv_id":"2312.05821","date":"2023-12-10","proceeding":null,"authors":["Zhihang Yuan","Yuzhang Shang","Yue Song","Qiang Wu","Yan Yan","Guangyu Sun"],"abstract":"In this paper, we introduce a new post-training compression paradigm for Large Language Models (LLMs) to facilitate their wider adoption. We delve into LLM weight low-rank decomposition, and find that the challenges of this task stem from the distribution variance in the LLM activations and the sensitivity difference among various kinds of layers. To address these issues, we propose a training-free approach called Activation-aware Singular Value Decomposition (ASVD). Specifically, ASVD manages activation outliers by transforming the weight matrix based on the activation distribution. This transformation allows the outliers in the activation matrix to be absorbed into the transformed weight matrix, thereby enhancing decomposition accuracy. Additionally, we propose an efficient iterative calibration process to optimize layer-specific decomposition by addressing the varying sensitivity of different LLM layers. In this way, ASVD can compress a network by 10%-30%. Based on the success of the low-rank decomposition of projection matrices in the self-attention module, we further introduce ASVD to compress the KV cache. By reducing the channel dimension of KV activations, memory requirements for KV cache can be largely reduced. ASVD can further achieve 50% KV cache reductions without performance drop in a training-free manner. Code is anonymously available in supplementary materials.","url_abs":"https://arxiv.org/abs/2312.05821v4","url_pdf":"https://arxiv.org/pdf/2312.05821v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"asvd-activation-aware-singular-value","repo_url":"https://github.com/hahnyuan/asvd4llm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"sensitivity","task_name":"Sensitivity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.05821","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.05821"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hahnyuan/asvd4llm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_draft_wrong":1,"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"65c7a547e405a20f","entry":"evaluate_perplexity","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"evaluate_utils.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/evaluate_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65c7a547e405a20f"}},{"code_sha256_prefix":"a9e7f2cdf016b88b","entry":"find_layers","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"quantization.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/quantization.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9e7f2cdf016b88b"}},{"code_sha256_prefix":"41c57e55d21c4d96","entry":"quantize","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"quantization.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/quantization.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41c57e55d21c4d96"}},{"code_sha256_prefix":"a2263a0b5d8a86f2","entry":"calib_sensitivity_stable_rank","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"sensitivity.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/sensitivity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2263a0b5d8a86f2"}},{"code_sha256_prefix":"1cbf0df5b5d26b2a","entry":"get_english_quote","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"datautils.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1cbf0df5b5d26b2a"}},{"code_sha256_prefix":"783b484325d75ee8","entry":"get_redpajama_train","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"datautils.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"783b484325d75ee8"}},{"code_sha256_prefix":"24c1cdae1808e107","entry":"sample_train_loaders","repo":"hahnyuan/asvd4llm","repo_kind":"official","path":"datautils.py","file_url":"https://github.com/hahnyuan/asvd4llm/blob/HEAD/datautils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24c1cdae1808e107"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}