{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/loftq-lora-fine-tuning-aware-quantization-for","title":"LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models","arxiv_id":"2310.08659","date":"2023-10-12","proceeding":null,"authors":["Yixiao Li","Yifan Yu","Chen Liang","Pengcheng He","Nikos Karampatziakis","Weizhu Chen","Tuo Zhao"],"abstract":"Quantization is an indispensable technique for serving Large Language Models (LLMs) and has recently found its way into LoRA fine-tuning. In this work we focus on the scenario where quantization and LoRA fine-tuning are applied together on a pre-trained model. In such cases it is common to observe a consistent gap in the performance on downstream tasks between full fine-tuning and quantization plus LoRA fine-tuning approach. In response, we propose LoftQ (LoRA-Fine-Tuning-aware Quantization), a novel quantization framework that simultaneously quantizes an LLM and finds a proper low-rank initialization for LoRA fine-tuning. Such an initialization alleviates the discrepancy between the quantized and full-precision model and significantly improves generalization in downstream tasks. We evaluate our method on natural language understanding, question answering, summarization, and natural language generation tasks. Experiments show that our method is highly effective and outperforms existing quantization methods, especially in the challenging 2-bit and 2/4-bit mixed precision regimes. The code is available on https://github.com/yxli2123/LoftQ.","url_abs":"https://arxiv.org/abs/2310.08659v4","url_pdf":"https://arxiv.org/pdf/2310.08659v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"loftq-lora-fine-tuning-aware-quantization-for","repo_url":"https://github.com/yxli2123/loftq","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.08659","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.08659"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yxli2123/loftq","reach":null}],"summary":{"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6d89eb39c2cd1646","entry":"compute_accuracy","repo":"yxli2123/loftq","repo_kind":"official","path":"test_gsm8k.py","file_url":"https://github.com/yxli2123/loftq/blob/HEAD/test_gsm8k.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d89eb39c2cd1646"}},{"code_sha256_prefix":"61d9388cf15f428e","entry":"extract_answer_number","repo":"yxli2123/loftq","repo_kind":"official","path":"test_gsm8k.py","file_url":"https://github.com/yxli2123/loftq/blob/HEAD/test_gsm8k.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61d9388cf15f428e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}