{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/roste-an-efficient-quantization-aware","title":"RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models","arxiv_id":"2502.09003","date":"2025-02-13","proceeding":null,"authors":["Quan Wei","Chung-Yiu Yau","Hoi-To Wai","Yang","Zhao","Dongyeop Kang","Youngsuk Park","Mingyi Hong"],"abstract":"Supervised fine-tuning is a standard method for adapting pre-trained large language models (LLMs) to downstream tasks. Quantization has been recently studied as a post-training technique for efficient LLM deployment. To obtain quantized fine-tuned LLMs, conventional pipelines would first fine-tune the pre-trained models, followed by post-training quantization. This often yields suboptimal performance as it fails to leverage the synergy between fine-tuning and quantization. To effectively realize low-bit quantization of weights, activations, and KV caches in LLMs, we propose an algorithm named Rotated Straight-Through-Estimator (RoSTE), which combines quantization-aware supervised fine-tuning (QA-SFT) with an adaptive rotation strategy that identifies an effective rotation configuration to reduce activation outliers. We provide theoretical insights on RoSTE by analyzing its prediction error when applied to an overparameterized least square quantized training problem. Our findings reveal that the prediction error is directly proportional to the quantization error of the converged weights, which can be effectively managed through an optimized rotation configuration. Experiments on Pythia and Llama models of different sizes demonstrate the effectiveness of RoSTE. Compared to existing post-SFT quantization baselines, our method consistently achieves superior performances across various tasks and different LLM architectures.","url_abs":"https://arxiv.org/abs/2502.09003v1","url_pdf":"https://arxiv.org/pdf/2502.09003v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"llama","method_name":"LLaMA"},{"method_slug":"pythia","method_name":"Pythia"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.09003","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.09003"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/OptimAI-Lab/RoSTE","reach":null}],"summary":{"ran":5},"by_repo_kind":{"found_in_text":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"6186a4865db634ec","entry":"QuantLinear","repo":"OptimAI-Lab/RoSTE","repo_kind":"found_in_text","path":"quant_utils/quant_r_linear.py","file_url":"https://github.com/OptimAI-Lab/RoSTE/blob/HEAD/quant_utils/quant_r_linear.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6186a4865db634ec"}},{"code_sha256_prefix":"8f05c88cb17c2fba","entry":"QuantLinear_R_up","repo":"OptimAI-Lab/RoSTE","repo_kind":"found_in_text","path":"quant_utils/quant_r_linear.py","file_url":"https://github.com/OptimAI-Lab/RoSTE/blob/HEAD/quant_utils/quant_r_linear.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f05c88cb17c2fba"}},{"code_sha256_prefix":"883a9f215ac7b06e","entry":"QuantizePerChannel","repo":"OptimAI-Lab/RoSTE","repo_kind":"found_in_text","path":"quant_utils/quant_r_linear.py","file_url":"https://github.com/OptimAI-Lab/RoSTE/blob/HEAD/quant_utils/quant_r_linear.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"883a9f215ac7b06e"}},{"code_sha256_prefix":"2e7e83f1a0c5ccc1","entry":"QuantizePerTensor","repo":"OptimAI-Lab/RoSTE","repo_kind":"found_in_text","path":"quant_utils/quant_r_linear.py","file_url":"https://github.com/OptimAI-Lab/RoSTE/blob/HEAD/quant_utils/quant_r_linear.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2e7e83f1a0c5ccc1"}},{"code_sha256_prefix":"2202b11bafc39d9f","entry":"QuantizePerToken","repo":"OptimAI-Lab/RoSTE","repo_kind":"found_in_text","path":"quant_utils/quant_r_linear.py","file_url":"https://github.com/OptimAI-Lab/RoSTE/blob/HEAD/quant_utils/quant_r_linear.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2202b11bafc39d9f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}