{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-simple-and-effective-pruning-approach-for","title":"A Simple and Effective Pruning Approach for Large Language Models","arxiv_id":"2306.11695","date":"2023-06-20","proceeding":null,"authors":["MingJie Sun","Zhuang Liu","Anna Bair","J. Zico Kolter"],"abstract":"As their size increases, Large Languages Models (LLMs) are natural candidates for network pruning methods: approaches that drop a subset of network weights while striving to preserve performance. Existing methods, however, require either retraining, which is rarely affordable for billion-scale LLMs, or solving a weight reconstruction problem reliant on second-order information, which may also be computationally expensive. In this paper, we introduce a novel, straightforward yet effective pruning method, termed Wanda (Pruning by Weights and activations), designed to induce sparsity in pretrained LLMs. Motivated by the recent observation of emergent large magnitude features in LLMs, our approach prunes weights with the smallest magnitudes multiplied by the corresponding input activations, on a per-output basis. Notably, Wanda requires no retraining or weight update, and the pruned LLM can be used as is. We conduct a thorough evaluation of our method Wanda on LLaMA and LLaMA-2 across various language benchmarks. Wanda significantly outperforms the established baseline of magnitude pruning and performs competitively against recent method involving intensive weight update. Code is available at https://github.com/locuslab/wanda.","url_abs":"https://arxiv.org/abs/2306.11695v3","url_pdf":"https://arxiv.org/pdf/2306.11695v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/locuslab/wanda","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/baithebest/adagp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/baithebest/sparsellm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/crystaleye42/eval-safety","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/nvlabs/maskllm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/qiaoxiao7282/seft","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"a-simple-and-effective-pruning-approach-for","repo_url":"https://github.com/ruchikachavhan/concept-prune","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.11695","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.11695"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/locuslab/wanda","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruchikachavhan/concept-prune","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baithebest/sparsellm","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baithebest/adagp","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nvlabs/maskllm","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qiaoxiao7282/seft","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/crystaleye42/eval-safety","reach":null}],"summary":{"ran":5,"ran_honours":1,"ran_draft_wrong":2,"ran_fixture":1,"unverified":13},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1},"listed":{"samples":15,"ran":6,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"f0bf77723cb645ec","entry":"BaseNeuronReceiver","repo":"ruchikachavhan/concept-prune","repo_kind":"listed","path":"neuron_receivers/wanda_receiver.py","file_url":"https://github.com/ruchikachavhan/concept-prune/blob/HEAD/neuron_receivers/wanda_receiver.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0bf77723cb645ec"}},{"code_sha256_prefix":"5a2c0dc15b39cddf","entry":"ColumnNormCalculator","repo":"ruchikachavhan/concept-prune","repo_kind":"listed","path":"neuron_receivers/wanda_receiver.py","file_url":"https://github.com/ruchikachavhan/concept-prune/blob/HEAD/neuron_receivers/wanda_receiver.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a2c0dc15b39cddf"}},{"code_sha256_prefix":"e36ff7e861167680","entry":"SparseGPT_LlaMA","repo":"baithebest/adagp","repo_kind":"listed","path":"pruning_utils.py","file_url":"https://github.com/baithebest/adagp/blob/HEAD/pruning_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e36ff7e861167680"}},{"code_sha256_prefix":"fbd6d90f750b7edc","entry":"TimeLayerColumnNorm","repo":"ruchikachavhan/concept-prune","repo_kind":"listed","path":"neuron_receivers/wanda_receiver.py","file_url":"https://github.com/ruchikachavhan/concept-prune/blob/HEAD/neuron_receivers/wanda_receiver.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbd6d90f750b7edc"}},{"code_sha256_prefix":"ac7406fce0dca1ac","entry":"Wanda","repo":"ruchikachavhan/concept-prune","repo_kind":"listed","path":"neuron_receivers/wanda_receiver.py","file_url":"https://github.com/ruchikachavhan/concept-prune/blob/HEAD/neuron_receivers/wanda_receiver.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac7406fce0dca1ac"}},{"code_sha256_prefix":"3b7b64b9aea9a039","entry":"check_sparsity","repo":"locuslab/wanda","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/locuslab/wanda/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b7b64b9aea9a039"}},{"code_sha256_prefix":"56ac52f94f2cc872","entry":"find_layers","repo":"locuslab/wanda","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/locuslab/wanda/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56ac52f94f2cc872"}},{"code_sha256_prefix":"d79ce728b3ab7682","entry":"return_given_alpha","repo":"crystaleye42/eval-safety","repo_kind":"listed","path":"lib/prune.py","file_url":"https://github.com/crystaleye42/eval-safety/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d79ce728b3ab7682"}},{"code_sha256_prefix":"02fe1aebf8355cab","entry":"sc","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"02fe1aebf8355cab"}},{"code_sha256_prefix":"9660cb0c932e836e","entry":"get_c4","repo":"crystaleye42/eval-safety","repo_kind":"listed","path":"lib/prune.py","file_url":"https://github.com/crystaleye42/eval-safety/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9660cb0c932e836e"}},{"code_sha256_prefix":"112fa6e5a362cc14","entry":"get_c4","repo":"qiaoxiao7282/seft","repo_kind":"listed","path":"lib/prune_all.py","file_url":"https://github.com/qiaoxiao7282/seft/blob/HEAD/lib/prune_all.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"112fa6e5a362cc14"}},{"code_sha256_prefix":"5a7a61a0235b3907","entry":"get_c4","repo":"locuslab/wanda","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/locuslab/wanda/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5a7a61a0235b3907"}},{"code_sha256_prefix":"142350989c8691e6","entry":"get_loaders","repo":"crystaleye42/eval-safety","repo_kind":"listed","path":"lib/prune.py","file_url":"https://github.com/crystaleye42/eval-safety/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"142350989c8691e6"}},{"code_sha256_prefix":"a0fc985586f0984c","entry":"get_loaders","repo":"qiaoxiao7282/seft","repo_kind":"listed","path":"lib/prune_all.py","file_url":"https://github.com/qiaoxiao7282/seft/blob/HEAD/lib/prune_all.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0fc985586f0984c"}},{"code_sha256_prefix":"d2a4a72ebbf33444","entry":"get_loaders","repo":"locuslab/wanda","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/locuslab/wanda/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2a4a72ebbf33444"}},{"code_sha256_prefix":"1afe3719e760f32e","entry":"get_wikitext2","repo":"crystaleye42/eval-safety","repo_kind":"listed","path":"lib/prune.py","file_url":"https://github.com/crystaleye42/eval-safety/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1afe3719e760f32e"}},{"code_sha256_prefix":"33272894b6721cfb","entry":"prepare_calibration_input","repo":"locuslab/wanda","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/locuslab/wanda/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33272894b6721cfb"}},{"code_sha256_prefix":"9aed9bf93575f59e","entry":"prepare_calibration_input","repo":"crystaleye42/eval-safety","repo_kind":"listed","path":"lib/prune.py","file_url":"https://github.com/crystaleye42/eval-safety/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9aed9bf93575f59e"}},{"code_sha256_prefix":"09d5f6f2798113c6","entry":"prepare_calibration_input","repo":"qiaoxiao7282/seft","repo_kind":"listed","path":"lib/prune_all.py","file_url":"https://github.com/qiaoxiao7282/seft/blob/HEAD/lib/prune_all.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09d5f6f2798113c6"}},{"code_sha256_prefix":"f4e32af5cd3a92c5","entry":"prune_wanda","repo":"crystaleye42/eval-safety","repo_kind":"listed","path":"lib/prune.py","file_url":"https://github.com/crystaleye42/eval-safety/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f4e32af5cd3a92c5"}},{"code_sha256_prefix":"3754399ccd7d5c49","entry":"prune_wanda","repo":"qiaoxiao7282/seft","repo_kind":"listed","path":"lib/prune_all.py","file_url":"https://github.com/qiaoxiao7282/seft/blob/HEAD/lib/prune_all.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3754399ccd7d5c49"}},{"code_sha256_prefix":"20569259cb1e7307","entry":"prune_wanda","repo":"locuslab/wanda","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/locuslab/wanda/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"20569259cb1e7307"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}