{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fluctuation-based-adaptive-structured-pruning","title":"Fluctuation-based Adaptive Structured Pruning for Large Language Models","arxiv_id":"2312.11983","date":"2023-12-19","proceeding":null,"authors":["Yongqi An","Xu Zhao","Tao Yu","Ming Tang","Jinqiao Wang"],"abstract":"Network Pruning is a promising way to address the huge computing resource demands of the deployment and inference of Large Language Models (LLMs). Retraining-free is important for LLMs' pruning methods. However, almost all of the existing retraining-free pruning approaches for LLMs focus on unstructured pruning, which requires specific hardware support for acceleration. In this paper, we propose a novel retraining-free structured pruning framework for LLMs, named FLAP (FLuctuation-based Adaptive Structured Pruning). It is hardware-friendly by effectively reducing storage and enhancing inference speed. For effective structured pruning of LLMs, we highlight three critical elements that demand the utmost attention: formulating structured importance metrics, adaptively searching the global compressed model, and implementing compensation mechanisms to mitigate performance loss. First, FLAP determines whether the output feature map is easily recoverable when a column of weight is removed, based on the fluctuation pruning metric. Then it standardizes the importance scores to adaptively determine the global compressed model structure. At last, FLAP adds additional bias terms to recover the output feature maps using the baseline values. We thoroughly evaluate our approach on a variety of language benchmarks. Without any retraining, our method significantly outperforms the state-of-the-art methods, including LLM-Pruner and the extension of Wanda in structured pruning. The code is released at https://github.com/CASIA-IVA-Lab/FLAP.","url_abs":"https://arxiv.org/abs/2312.11983v1","url_pdf":"https://arxiv.org/pdf/2312.11983v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fluctuation-based-adaptive-structured-pruning","repo_url":"https://github.com/casia-iva-lab/flap","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.11983","atlas_url":"https://app.syntology.ai/?focus=2312.11983","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.11983"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/casia-iva-lab/flap","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"ran":1,"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":8,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9b4dff79d5e6102c","entry":"apply_rotary_pos_emb","repo":"casia-iva-lab/flap","repo_kind":"official","path":"models/hf_llama/modeling_llama.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/models/hf_llama/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9b4dff79d5e6102c"}},{"code_sha256_prefix":"d7befe8027d275aa","entry":"check_sparsity","repo":"casia-iva-lab/flap","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/lib/prune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d7befe8027d275aa"}},{"code_sha256_prefix":"2ed16fcdc14ad951","entry":"find_layers","repo":"casia-iva-lab/flap","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/lib/prune.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ed16fcdc14ad951"}},{"code_sha256_prefix":"b99eea6376d1e212","entry":"rotate_half","repo":"casia-iva-lab/flap","repo_kind":"official","path":"models/hf_llama/modeling_llama.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/models/hf_llama/modeling_llama.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b99eea6376d1e212"}},{"code_sha256_prefix":"6ccf0fe3fd4574d5","entry":"get_c4","repo":"casia-iva-lab/flap","repo_kind":"official","path":"lib/data.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/lib/data.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6ccf0fe3fd4574d5"}},{"code_sha256_prefix":"1d035152616b1d19","entry":"get_ptb","repo":"casia-iva-lab/flap","repo_kind":"official","path":"lib/data.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/lib/data.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d035152616b1d19"}},{"code_sha256_prefix":"2d79e7ef9f571f3d","entry":"get_wikitext2","repo":"casia-iva-lab/flap","repo_kind":"official","path":"lib/data.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/lib/data.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d79e7ef9f571f3d"}},{"code_sha256_prefix":"2519b3efd481ad47","entry":"prepare_calibration_input","repo":"casia-iva-lab/flap","repo_kind":"official","path":"lib/prune.py","file_url":"https://github.com/casia-iva-lab/flap/blob/HEAD/lib/prune.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2519b3efd481ad47"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}