{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-gradient-flow-framework-for-analyzing","title":"A Gradient Flow Framework For Analyzing Network Pruning","arxiv_id":"2009.11839","date":"2020-09-24","proceeding":"ICLR 2021 1","authors":["Ekdeep Singh Lubana","Robert P. Dick"],"abstract":"Recent network pruning methods focus on pruning models early-on in training. To estimate the impact of removing a parameter, these methods use importance measures that were originally designed to prune trained models. Despite lacking justification for their use early-on in training, such measures result in surprisingly low accuracy loss. To better explain this behavior, we develop a general framework that uses gradient flow to unify state-of-the-art importance measures through the norm of model parameters. We use this framework to determine the relationship between pruning measures and evolution of model parameters, establishing several results related to pruning models early-on in training: (i) magnitude-based pruning removes parameters that contribute least to reduction in loss, resulting in models that converge faster than magnitude-agnostic methods; (ii) loss-preservation based pruning preserves first-order model evolution dynamics and is therefore appropriate for pruning minimally trained models; and (iii) gradient-norm based pruning affects second-order model evolution dynamics, such that increasing gradient norm via pruning can produce poorly performing models. We validate our claims on several VGG-13, MobileNet-V1, and ResNet-56 models trained on CIFAR-10/CIFAR-100. Code available at https://github.com/EkdeepSLubana/flowandprune.","url_abs":"https://arxiv.org/abs/2009.11839v4","url_pdf":"https://arxiv.org/pdf/2009.11839v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-gradient-flow-framework-for-analyzing","repo_url":"https://github.com/EkdeepSLubana/flowandprune","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2009.11839","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2009.11839"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EkdeepSLubana/flowandprune","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a207bfea1cc1de3a","entry":"model_grads","repo":"EkdeepSLubana/flowandprune","repo_kind":"official","path":"imp_estimator.py","file_url":"https://github.com/EkdeepSLubana/flowandprune/blob/HEAD/imp_estimator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a207bfea1cc1de3a"}},{"code_sha256_prefix":"81aa0dbfe708b587","entry":"model_params","repo":"EkdeepSLubana/flowandprune","repo_kind":"official","path":"imp_estimator.py","file_url":"https://github.com/EkdeepSLubana/flowandprune/blob/HEAD/imp_estimator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81aa0dbfe708b587"}},{"code_sha256_prefix":"be3d3646a3c9291a","entry":"cal_hg","repo":"EkdeepSLubana/flowandprune","repo_kind":"official","path":"imp_estimator.py","file_url":"https://github.com/EkdeepSLubana/flowandprune/blob/HEAD/imp_estimator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be3d3646a3c9291a"}},{"code_sha256_prefix":"695909300c407d17","entry":"cal_importance_grasp","repo":"EkdeepSLubana/flowandprune","repo_kind":"official","path":"imp_estimator.py","file_url":"https://github.com/EkdeepSLubana/flowandprune/blob/HEAD/imp_estimator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"695909300c407d17"}},{"code_sha256_prefix":"80ac3c8529d510c3","entry":"grasp_data","repo":"EkdeepSLubana/flowandprune","repo_kind":"official","path":"imp_estimator.py","file_url":"https://github.com/EkdeepSLubana/flowandprune/blob/HEAD/imp_estimator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80ac3c8529d510c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}