{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/do-emergent-abilities-exist-in-quantized","title":"Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study","arxiv_id":"2307.08072","date":"2023-07-16","proceeding":null,"authors":["Peiyu Liu","Zikang Liu","Ze-Feng Gao","Dawei Gao","Wayne Xin Zhao","Yaliang Li","Bolin Ding","Ji-Rong Wen"],"abstract":"Despite the superior performance, Large Language Models~(LLMs) require significant computational resources for deployment and use. To overcome this issue, quantization methods have been widely applied to reduce the memory footprint of LLMs as well as increasing the inference rate. However, a major challenge is that low-bit quantization methods often lead to performance degradation. It is important to understand how quantization impacts the capacity of LLMs. Different from previous studies focused on overall performance, this work aims to investigate the impact of quantization on \\emph{emergent abilities}, which are important characteristics that distinguish LLMs from small language models. Specially, we examine the abilities of in-context learning, chain-of-thought reasoning, and instruction-following in quantized LLMs. Our empirical experiments show that these emergent abilities still exist in 4-bit quantization models, while 2-bit models encounter severe performance degradation on the test of these abilities. To improve the performance of low-bit models, we conduct two special experiments: (1) fine-gained impact analysis that studies which components (or substructures) are more sensitive to quantization, and (2) performance compensation through model fine-tuning. Our work derives a series of important findings to understand the impact of quantization on emergent abilities, and sheds lights on the possibilities of extremely low-bit quantization for LLMs.","url_abs":"https://arxiv.org/abs/2307.08072v2","url_pdf":"https://arxiv.org/pdf/2307.08072v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"do-emergent-abilities-exist-in-quantized","repo_url":"https://github.com/rucaibox/quantizedempirical","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.08072","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.08072"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rucaibox/quantizedempirical","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":5,"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":14,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"30d7eec482ebf6b1","entry":"repeat_kv","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/modeling_llama2.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/modeling_llama2.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30d7eec482ebf6b1"}},{"code_sha256_prefix":"f725bc2d76076485","entry":"apply_rotary_pos_emb","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/modeling_llama2.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/modeling_llama2.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f725bc2d76076485"}},{"code_sha256_prefix":"373a7df152e5a1a5","entry":"apply_rotary_pos_emb","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/modeling_llama.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"373a7df152e5a1a5"}},{"code_sha256_prefix":"507f475feb734647","entry":"compute_intermediate_size","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"convert_llama_weights_to_hf.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/convert_llama_weights_to_hf.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"507f475feb734647"}},{"code_sha256_prefix":"0c540da8d3df6b6e","entry":"compute_intermediate_size","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/convert_llama_weights_to_hf.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/convert_llama_weights_to_hf.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c540da8d3df6b6e"}},{"code_sha256_prefix":"7c76c83fca14fe69","entry":"cpu_offload","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/bigmodeling.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/bigmodeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7c76c83fca14fe69"}},{"code_sha256_prefix":"d697b8d42cbb73ba","entry":"cpu_offload_with_hook","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/bigmodeling.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/bigmodeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d697b8d42cbb73ba"}},{"code_sha256_prefix":"56e674752db00649","entry":"generate_prompt","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/finetune.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56e674752db00649"}},{"code_sha256_prefix":"2c201f017173954b","entry":"get_llama","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/llama.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/llama.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c201f017173954b"}},{"code_sha256_prefix":"77981ab8dcab7d60","entry":"init_on_device","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/bigmodeling.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/bigmodeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77981ab8dcab7d60"}},{"code_sha256_prefix":"c5bcf01d18bba63d","entry":"read_json","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"convert_llama_weights_to_hf.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/convert_llama_weights_to_hf.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5bcf01d18bba63d"}},{"code_sha256_prefix":"b99eea6376d1e212","entry":"rotate_half","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/modeling_llama.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b99eea6376d1e212"}},{"code_sha256_prefix":"25ab7cdc2eb3c237","entry":"write_model","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"convert_llama_weights_to_hf.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/convert_llama_weights_to_hf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25ab7cdc2eb3c237"}},{"code_sha256_prefix":"57f5bf024e9e6eb0","entry":"write_model","repo":"rucaibox/quantizedempirical","repo_kind":"official","path":"models/convert_llama_weights_to_hf.py","file_url":"https://github.com/rucaibox/quantizedempirical/blob/HEAD/models/convert_llama_weights_to_hf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57f5bf024e9e6eb0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}