{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/isomorphic-pruning-for-vision-models","title":"Isomorphic Pruning for Vision Models","arxiv_id":"2407.04616","date":"2024-07-05","proceeding":null,"authors":["Gongfan Fang","Xinyin Ma","Michael Bi Mi","Xinchao Wang"],"abstract":"Structured pruning reduces the computational overhead of deep neural networks by removing redundant sub-structures. However, assessing the relative importance of different sub-structures remains a significant challenge, particularly in advanced vision models featuring novel mechanisms and architectures like self-attention, depth-wise convolutions, or residual connections. These heterogeneous substructures usually exhibit diverged parameter scales, weight distributions, and computational topology, introducing considerable difficulty to importance comparison. To overcome this, we present Isomorphic Pruning, a simple approach that demonstrates effectiveness across a range of network architectures such as Vision Transformers and CNNs, and delivers competitive performance across different model sizes. Isomorphic Pruning originates from an observation that, when evaluated under a pre-defined importance criterion, heterogeneous sub-structures demonstrate significant divergence in their importance distribution, as opposed to isomorphic structures that present similar importance patterns. This inspires us to perform isolated ranking and comparison on different types of sub-structures for more reliable pruning. Our empirical results on ImageNet-1K demonstrate that Isomorphic Pruning surpasses several pruning baselines dedicatedly designed for Transformers or CNNs. For instance, we improve the accuracy of DeiT-Tiny from 74.52% to 77.50% by pruning an off-the-shelf DeiT-Base model. And for ConvNext-Tiny, we enhanced performance from 82.06% to 82.18%, while reducing the number of parameters and memory usage. Code is available at \\url{https://github.com/VainF/Isomorphic-Pruning}.","url_abs":"https://arxiv.org/abs/2407.04616v1","url_pdf":"https://arxiv.org/pdf/2407.04616v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"isomorphic-pruning-for-vision-models","repo_url":"https://github.com/vainf/isomorphic-pruning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.04616","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.04616"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vainf/isomorphic-pruning","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/VainF/Isomorphic-Pruning","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":2,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"0d95e28fac98427f","entry":"accuracy","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"pbench/utils.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/pbench/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d95e28fac98427f"}},{"code_sha256_prefix":"89e2ddf547872100","entry":"validate","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"convnext_train.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/convnext_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89e2ddf547872100"}},{"code_sha256_prefix":"4627b6cf43e4005f","entry":"validate_model","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4627b6cf43e4005f"}},{"code_sha256_prefix":"4eca7815147e1708","entry":"average_checkpoints","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"pbench/utils.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/pbench/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4eca7815147e1708"}},{"code_sha256_prefix":"9ff9b233192c1093","entry":"resume_checkpoint","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"convnext_train.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/convnext_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ff9b233192c1093"}},{"code_sha256_prefix":"d92101f375faa8a6","entry":"store_model_weights","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"pbench/utils.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/pbench/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d92101f375faa8a6"}},{"code_sha256_prefix":"83f2bf35d7803cda","entry":"train_one_epoch","repo":"VainF/Isomorphic-Pruning","repo_kind":"official","path":"convnext_train.py","file_url":"https://github.com/VainF/Isomorphic-Pruning/blob/HEAD/convnext_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"83f2bf35d7803cda"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}