{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-00494","title":"ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression","arxiv_id":"2606.00494","date":"2026-05-30","proceeding":"ICML","authors":["Wenya Yu","Chao Zhang","Li Wang","Samson Lasaulce","Merouane Debbah"],"abstract":"Post-Training Quantization (PTQ) and Low-Rank Adaptation (LoRA) constitute the standard pipeline for efficient Large Language Model (LLM) deployment. However, applying them sequentially poses a problem: PTQ often leaves behind random noise that is spread out (across the model's weights) in a way LoRA can't easily fix, meaning that LoRA ends up wasting its limited capacity trying to fix uncorrectable noise instead of improving task performance. In this paper, we propose \\textbf{ProjQ}, a novel framework for constraining quantization noise to the low-rank manifold via orthogonal subspace projection. We derive an efficient alternating algorithm that shapes the quantization noise into a low-rank structure, effectively offloading dominant error components to the subsequent adapter while minimizing the residual error in the orthogonal \"uncorrectable\" subspace. Our theoretical analysis demonstrates that ProjQ preserves strictly greater model plasticity for downstream tasks compared to standard PTQ. Extensive experiments on LLaMA-2, Qwen2.5 and Qwen3 confirm that ProjQ consistently outperforms existing methods in both quantization error compensation and downstream task fine-tuning, achieving up to $2\\times$ lower evaluation loss for compensation and matching the performance of standard 4-bit baselines on language modeling tasks with only 3 bits. The code is available on https://github.com/yy9301/ProjQ .","url_abs":"https://arxiv.org/abs/2606.00494","url_pdf":"https://arxiv.org/pdf/2606.00494","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.00494","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.00494"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/yy9301/ProjQ","reach":null}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"e27a117c978bd39d","entry":"eval_lambada","repo":"yy9301/ProjQ","repo_kind":"found_in_text","path":"eval_acc.py","file_url":"https://github.com/yy9301/ProjQ/blob/HEAD/eval_acc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e27a117c978bd39d"}},{"code_sha256_prefix":"d313d28375a69ab9","entry":"get_ptb","repo":"yy9301/ProjQ","repo_kind":"found_in_text","path":"datautils.py","file_url":"https://github.com/yy9301/ProjQ/blob/HEAD/datautils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d313d28375a69ab9"}},{"code_sha256_prefix":"a58b0d6a4432b5d0","entry":"piqa_context","repo":"yy9301/ProjQ","repo_kind":"found_in_text","path":"comp_train.py","file_url":"https://github.com/yy9301/ProjQ/blob/HEAD/comp_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a58b0d6a4432b5d0"}},{"code_sha256_prefix":"27d9e743720c252f","entry":"eval_arc_C","repo":"yy9301/ProjQ","repo_kind":"found_in_text","path":"eval_acc.py","file_url":"https://github.com/yy9301/ProjQ/blob/HEAD/eval_acc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27d9e743720c252f"}},{"code_sha256_prefix":"7d35663de73e6b0b","entry":"eval_arc_E","repo":"yy9301/ProjQ","repo_kind":"found_in_text","path":"eval_acc.py","file_url":"https://github.com/yy9301/ProjQ/blob/HEAD/eval_acc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d35663de73e6b0b"}},{"code_sha256_prefix":"eba3002eddf452e0","entry":"get_number_of_rows_and_cols","repo":"yy9301/ProjQ","repo_kind":"found_in_text","path":"gptqmodel/quantization/projq_gptq.py","file_url":"https://github.com/yy9301/ProjQ/blob/HEAD/gptqmodel/quantization/projq_gptq.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eba3002eddf452e0"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}