{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/depgraph-towards-any-structural-pruning","title":"DepGraph: Towards Any Structural Pruning","arxiv_id":"2301.12900","date":"2023-01-30","proceeding":"CVPR 2023 1","authors":["Gongfan Fang","Xinyin Ma","Mingli Song","Michael Bi Mi","Xinchao Wang"],"abstract":"Structural pruning enables model acceleration by removing structurally-grouped parameters from neural networks. However, the parameter-grouping patterns vary widely across different models, making architecture-specific pruners, which rely on manually-designed grouping schemes, non-generalizable to new architectures. In this work, we study a highly-challenging yet barely-explored task, any structural pruning, to tackle general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. The most prominent obstacle towards this goal lies in the structural coupling, which not only forces different layers to be pruned simultaneously, but also expects all removed parameters to be consistently unimportant, thereby avoiding structural issues and significant performance degradation after pruning. To address this problem, we propose a general and {fully automatic} method, \\emph{Dependency Graph} (DepGraph), to explicitly model the dependency between layers and comprehensively group coupled parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a simple norm-based criterion, the proposed method consistently yields gratifying performances.","url_abs":"https://arxiv.org/abs/2301.12900v2","url_pdf":"https://arxiv.org/pdf/2301.12900v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"depgraph-towards-any-structural-pruning","repo_url":"https://github.com/VainF/Torch-Pruning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"network-pruning","task_name":"Network Pruning"},{"task_slug":"neural-network-compression","task_name":"Neural Network Compression"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dgcnn","method_name":"DGCNN"},{"method_slug":"dense-block","method_name":"Dense Block"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gat","method_name":"GAT"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2301.12900","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.12900"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VainF/Torch-Pruning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":6},"by_repo_kind":{"official":{"samples":7,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"915b7da0362eb2b4","entry":"is_scalar","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/_helpers.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/_helpers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"915b7da0362eb2b4"}},{"code_sha256_prefix":"44927cfc49a45803","entry":"load_state_dict","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/serialization.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/serialization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"44927cfc49a45803"}},{"code_sha256_prefix":"499a656713eb0e7a","entry":"module2type","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/ops.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"499a656713eb0e7a"}},{"code_sha256_prefix":"f8d1be76d0300bbf","entry":"state_dict","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/serialization.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/serialization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8d1be76d0300bbf"}},{"code_sha256_prefix":"a5f40ecfcfcbe7c1","entry":"to_plain_idxs","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/_helpers.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/_helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5f40ecfcfcbe7c1"}},{"code_sha256_prefix":"b4c2def331fc3065","entry":"to_root_idxs","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/_helpers.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/_helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4c2def331fc3065"}},{"code_sha256_prefix":"33b3f48ffee9c0cd","entry":"type2class","repo":"VainF/Torch-Pruning","repo_kind":"official","path":"torch_pruning/ops.py","file_url":"https://github.com/VainF/Torch-Pruning/blob/HEAD/torch_pruning/ops.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33b3f48ffee9c0cd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}