{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-07955","title":"Rethinking Residual Errors in Compensation-based LLM Quantization","arxiv_id":"2604.07955","date":"2026-04-09","proceeding":null,"authors":["Shuaiting Li","Juncan Deng","Kedong Xu","Rongtao Deng","Hong Gu","Minghan Jiang","Haibin Shen","Kejie Huang"],"abstract":"Methods based on weight compensation, which iteratively apply quantization and weight compensation to minimize the output error, have recently demonstrated remarkable success in quantizing Large Language Models (LLMs). The representative work, GPTQ, introduces several key techniques that make such iterative methods practical for LLMs with billions of parameters. GPTAQ extends this approach by introducing an asymmetric calibration process that aligns the output of each quantized layer with its full-precision counterpart, incorporating a residual error into the weight compensation framework. In this work, we revisit the formulation of the residual error. We identify a sub-optimal calibration objective in existing methods: during the intra-layer calibration process, they align the quantized output with the output from compensated weights, rather than the true output from the original full-precision model. Therefore, we redefine the objective to precisely align the quantized model's output with the original output of the full-precision model at each step. We then reveal that the residual error originates not only from the output difference of the preceding layer but also from the discrepancy between the compensated and original weights within each layer, which we name the 'compensation-aware error'. By inheriting the neuron decomposition technique from GPTAQ, we can efficiently incorporate this compensation-aware error into the weight update process. Extensive experiments on various LLMs and quantization settings demonstrate that our proposed enhancements integrate seamlessly with both GPTQ and GPTAQ, significantly improving their quantization performance. Our code is publicly available at https://github.com/list0830/ResComp.","url_abs":"https://arxiv.org/abs/2604.07955","url_pdf":"https://arxiv.org/pdf/2604.07955","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2604.07955","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.07955"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/list0830/ResComp","reach":null}],"summary":{"ran":3,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":8,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"8005fc23ccbe61fb","entry":"get_rope_function_name","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/model_utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/model_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8005fc23ccbe61fb"}},{"code_sha256_prefix":"fe39dd44b7d2698b","entry":"llama_down_proj_groupsize","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe39dd44b7d2698b"}},{"code_sha256_prefix":"522f6e6438fa5941","entry":"random_orthogonal_matrix","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/rotation_utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/rotation_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"522f6e6438fa5941"}},{"code_sha256_prefix":"88c8d591b4688832","entry":"cleanup_memory","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"88c8d591b4688832"}},{"code_sha256_prefix":"cf7f28fedcdd19c5","entry":"config_logging","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cf7f28fedcdd19c5"}},{"code_sha256_prefix":"eb59d0a73734e449","entry":"get_layers","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/model_utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/model_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb59d0a73734e449"}},{"code_sha256_prefix":"347f7e0a0b0e4f2f","entry":"get_wikitext2","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/data_utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"347f7e0a0b0e4f2f"}},{"code_sha256_prefix":"cdd805f7fc7a0842","entry":"model_type_extractor","repo":"list0830/ResComp","repo_kind":"found_in_text","path":"fake_quant/model_utils.py","file_url":"https://github.com/list0830/ResComp/blob/HEAD/fake_quant/model_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cdd805f7fc7a0842"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}