{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-empirical-study-of-qwen3-quantization","title":"An Empirical Study of Qwen3 Quantization","arxiv_id":"2505.02214","date":"2025-05-04","proceeding":null,"authors":["Xingyu Zheng","Yuye Li","Haoran Chu","Yue Feng","Xudong Ma","Jie Luo","Jinyang Guo","Haotong Qin","Michele Magno","Xianglong Liu"],"abstract":"The Qwen series has emerged as a leading family of open-source Large Language Models (LLMs), demonstrating remarkable capabilities in natural language understanding tasks. With the recent release of Qwen3, which exhibits superior performance across diverse benchmarks, there is growing interest in deploying these models efficiently in resource-constrained environments. Low-bit quantization presents a promising solution, yet its impact on Qwen3's performance remains underexplored. This study conducts a systematic evaluation of Qwen3's robustness under various quantization settings, aiming to uncover both opportunities and challenges in compressing this state-of-the-art model. We rigorously assess 5 existing classic post-training quantization techniques applied to Qwen3, spanning bit-widths from 1 to 8 bits, and evaluate their effectiveness across multiple datasets. Our findings reveal that while Qwen3 maintains competitive performance at moderate bit-widths, it experiences notable degradation in linguistic tasks under ultra-low precision, underscoring the persistent hurdles in LLM compression. These results emphasize the need for further research to mitigate performance loss in extreme quantization scenarios. We anticipate that this empirical analysis will provide actionable insights for advancing quantization methods tailored to Qwen3 and future LLMs, ultimately enhancing their practicality without compromising accuracy. Our project is released on https://github.com/Efficient-ML/Qwen3-Quantization and https://huggingface.co/collections/Efficient-ML/qwen3-quantization-68164450decb1c868788cb2b.","url_abs":"https://arxiv.org/abs/2505.02214v1","url_pdf":"https://arxiv.org/pdf/2505.02214v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-empirical-study-of-qwen3-quantization","repo_url":"https://github.com/efficient-ml/qwen3-quantization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.02214","atlas_url":"https://app.syntology.ai/?focus=2505.02214","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.02214"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/efficient-ml/qwen3-quantization","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":4,"ran":3,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a9e7f2cdf016b88b","entry":"find_layers","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/modelutils.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/modelutils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9e7f2cdf016b88b"}},{"code_sha256_prefix":"7ae3b57ecdeec075","entry":"high_order_residual","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/binary.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/binary.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ae3b57ecdeec075"}},{"code_sha256_prefix":"dd1edfd05dda7353","entry":"normal_quantize","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/binary.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/binary.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd1edfd05dda7353"}},{"code_sha256_prefix":"25eb3a441763f260","entry":"part_mean","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/binary.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/binary.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"25eb3a441763f260"}},{"code_sha256_prefix":"c1cb961b4f5cb75c","entry":"quantize_activation_per_token_absmax","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/fake_quant.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/fake_quant.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c1cb961b4f5cb75c"}},{"code_sha256_prefix":"770da8497b2ff4ba","entry":"quantize_weight_per_channel_absmax","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/fake_quant.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/fake_quant.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"770da8497b2ff4ba"}},{"code_sha256_prefix":"dd3493503cb95888","entry":"quantize_weight_per_tensor_absmax","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/fake_quant.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/fake_quant.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd3493503cb95888"}},{"code_sha256_prefix":"3fed3338f2ffb08f","entry":"get_act_scales","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/calibration.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/calibration.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3fed3338f2ffb08f"}},{"code_sha256_prefix":"a7cbf4fc49c71e03","entry":"get_ptb","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/datautils.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/datautils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a7cbf4fc49c71e03"}},{"code_sha256_prefix":"a5a75a01c343fb3b","entry":"get_static_decoder_layer_scales","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/calibration.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/SmoothQuant-for-Qwen3/smoothquant-main/smoothquant/calibration.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a5a75a01c343fb3b"}},{"code_sha256_prefix":"729f957048af4a3f","entry":"get_tokenizer","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/datautils.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/datautils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"729f957048af4a3f"}},{"code_sha256_prefix":"011909315dfdbc2e","entry":"get_wikitext2","repo":"efficient-ml/qwen3-quantization","repo_kind":"official","path":"BiLLM/datautils.py","file_url":"https://github.com/efficient-ml/qwen3-quantization/blob/HEAD/BiLLM/datautils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"011909315dfdbc2e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}