{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/quantized-side-tuning-fast-and-memory","title":"Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models","arxiv_id":"2401.07159","date":"2024-01-13","proceeding":null,"authors":["Zhengxin Zhang","Dan Zhao","Xupeng Miao","Gabriele Oliaro","Qing Li","Yong Jiang","Zhihao Jia"],"abstract":"Finetuning large language models (LLMs) has been empirically effective on a variety of downstream tasks. Existing approaches to finetuning an LLM either focus on parameter-efficient finetuning, which only updates a small number of trainable parameters, or attempt to reduce the memory footprint during the training phase of the finetuning. Typically, the memory footprint during finetuning stems from three contributors: model weights, optimizer states, and intermediate activations. However, existing works still require considerable memory and none can simultaneously mitigate memory footprint for all three sources. In this paper, we present Quantized Side Tuing (QST), which enables memory-efficient and fast finetuning of LLMs by operating through a dual-stage process. First, QST quantizes an LLM's model weights into 4-bit to reduce the memory footprint of the LLM's original weights; QST also introduces a side network separated from the LLM, which utilizes the hidden states of the LLM to make task-specific predictions. Using a separate side network avoids performing backpropagation through the LLM, thus reducing the memory requirement of the intermediate activations. Furthermore, QST leverages several low-rank adaptors and gradient-free downsample modules to significantly reduce the trainable parameters, so as to save the memory footprint of the optimizer states. Experiments show that QST can reduce the total memory footprint by up to 2.3 $\\times$ and speed up the finetuning process by up to 3 $\\times$ while achieving competent performance compared with the state-of-the-art. When it comes to full finetuning, QST can reduce the total memory footprint up to 7 $\\times$.","url_abs":"https://arxiv.org/abs/2401.07159v1","url_pdf":"https://arxiv.org/pdf/2401.07159v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"quantized-side-tuning-fast-and-memory","repo_url":"https://github.com/youarespecialtome/qst","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.07159","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.07159"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/youarespecialtome/qst","reach":null}],"summary":{"ran":4,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":10,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"62859370cfe52b3c","entry":"Activations","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62859370cfe52b3c"}},{"code_sha256_prefix":"f66240187e4c4ad0","entry":"AdapterLinear","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f66240187e4c4ad0"}},{"code_sha256_prefix":"6ef27137c2104cdd","entry":"LlamaDecoderLayer","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6ef27137c2104cdd"}},{"code_sha256_prefix":"393706ac28a3215a","entry":"LlamaMLP","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"393706ac28a3215a"}},{"code_sha256_prefix":"3c9f7e87051e9272","entry":"apply_rotary_pos_emb","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3c9f7e87051e9272"}},{"code_sha256_prefix":"49bff92fc40bac47","entry":"LlamaAttention","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"49bff92fc40bac47"}},{"code_sha256_prefix":"e4df1ae4f39faa71","entry":"LlamaModel","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e4df1ae4f39faa71"}},{"code_sha256_prefix":"c394ff56984f4c10","entry":"QSTBaseModelOutputWithPast","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c394ff56984f4c10"}},{"code_sha256_prefix":"fb6c4a667149f2a3","entry":"QSTConfig","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb6c4a667149f2a3"}},{"code_sha256_prefix":"4410d9c227d25d07","entry":"QSTLlamaModel","repo":"youarespecialtome/qst","repo_kind":"official","path":"modeling_llama_qst.py","file_url":"https://github.com/youarespecialtome/qst/blob/HEAD/modeling_llama_qst.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4410d9c227d25d07"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}