{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-pruning-via-growing-regularization-1","title":"Neural Pruning via Growing Regularization","arxiv_id":"2012.09243","date":"2020-12-16","proceeding":"ICLR 2021 1","authors":["Huan Wang","Can Qin","Yulun Zhang","Yun Fu"],"abstract":"Regularization has long been utilized to learn sparsity in deep neural network pruning. However, its role is mainly explored in the small penalty strength regime. In this work, we extend its application to a new scenario where the regularization grows large gradually to tackle two central problems of pruning: pruning schedule and weight importance scoring. (1) The former topic is newly brought up in this work, which we find critical to the pruning performance while receives little research attention. Specifically, we propose an L2 regularization variant with rising penalty factors and show it can bring significant accuracy gains compared with its one-shot counterpart, even when the same weights are removed. (2) The growing penalty scheme also brings us an approach to exploit the Hessian information for more accurate pruning without knowing their specific values, thus not bothered by the common Hessian approximation problems. Empirically, the proposed algorithms are easy to implement and scalable to large datasets and networks in both structured and unstructured pruning. Their effectiveness is demonstrated with modern deep neural networks on the CIFAR and ImageNet datasets, achieving competitive results compared to many state-of-the-art algorithms. Our code and trained models are publicly available at https://github.com/mingsuntse/regularization-pruning.","url_abs":"https://arxiv.org/abs/2012.09243v2","url_pdf":"https://arxiv.org/pdf/2012.09243v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-pruning-via-growing-regularization-1","repo_url":"https://github.com/mingsun-tse/regularization-pruning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"l2-regularization","task_name":"L2 Regularization"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.09243","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.09243"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/mingsuntse/regularization-pruning","reach":{"status":"unanswered"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Ericmingjie/rethinking-network-pruning","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mingsun-tse/regularization-pruning","reach":null}],"summary":{"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"d54cd5c9b293f300","entry":"Layer","repo":"mingsun-tse/regularization-pruning","repo_kind":"official","path":"pruner/reg_pruner.py","file_url":"https://github.com/mingsun-tse/regularization-pruning/blob/HEAD/pruner/reg_pruner.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d54cd5c9b293f300"}},{"code_sha256_prefix":"e67edd61de30baae","entry":"MetaPruner","repo":"mingsun-tse/regularization-pruning","repo_kind":"official","path":"pruner/reg_pruner.py","file_url":"https://github.com/mingsun-tse/regularization-pruning/blob/HEAD/pruner/reg_pruner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e67edd61de30baae"}},{"code_sha256_prefix":"3d649921f02ecc11","entry":"Pruner","repo":"mingsun-tse/regularization-pruning","repo_kind":"official","path":"pruner/reg_pruner.py","file_url":"https://github.com/mingsun-tse/regularization-pruning/blob/HEAD/pruner/reg_pruner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d649921f02ecc11"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}