{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-structured-sparsity-in-deep-neural","title":"Learning Structured Sparsity in Deep Neural Networks","arxiv_id":"1608.03665","date":"2016-08-12","proceeding":"NeurIPS 2016 12","authors":["Wei Wen","Chunpeng Wu","Yandan Wang","Yiran Chen","Hai Li"],"abstract":"High demand for computation resources severely hinders deployment of\nlarge-scale Deep Neural Networks (DNN) in resource constrained devices. In this\nwork, we propose a Structured Sparsity Learning (SSL) method to regularize the\nstructures (i.e., filters, channels, filter shapes, and layer depth) of DNNs.\nSSL can: (1) learn a compact structure from a bigger DNN to reduce computation\ncost; (2) obtain a hardware-friendly structured sparsity of DNN to efficiently\naccelerate the DNNs evaluation. Experimental results show that SSL achieves on\naverage 5.1x and 3.1x speedups of convolutional layer computation of AlexNet\nagainst CPU and GPU, respectively, with off-the-shelf libraries. These speedups\nare about twice speedups of non-structured sparsity; (3) regularize the DNN\nstructure to improve classification accuracy. The results show that for\nCIFAR-10, regularization on layer depth can reduce 20 layers of a Deep Residual\nNetwork (ResNet) to 18 layers while improve the accuracy from 91.25% to 92.60%,\nwhich is still slightly higher than that of original ResNet with 32 layers. For\nAlexNet, structure regularization by SSL also reduces the error by around ~1%.\nOpen source code is in https://github.com/wenwei202/caffe/tree/scnn","url_abs":"http://arxiv.org/abs/1608.03665v4","url_pdf":"http://arxiv.org/pdf/1608.03665v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-structured-sparsity-in-deep-neural","repo_url":"https://github.com/wenwei202/caffe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"learning-structured-sparsity-in-deep-neural","repo_url":"https://github.com/CEREBUSTER/effacinto","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"learning-structured-sparsity-in-deep-neural","repo_url":"https://github.com/ZJCV/SSL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":null,"task_name":"GPU"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1608.03665","atlas_url":"https://app.syntology.ai/?focus=1608.03665","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1608.03665"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wenwei202/caffe","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CEREBUSTER/effacinto","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZJCV/SSL","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":14},"by_repo_kind":{"listed":{"samples":14,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7161254df5577f1d","entry":"compute_mask","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_filter_and_channel.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_filter_and_channel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7161254df5577f1d"}},{"code_sha256_prefix":"6f0c98250fa6667e","entry":"computer_bottleneck_weight","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_resnet_by_depth.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_resnet_by_depth.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6f0c98250fa6667e"}},{"code_sha256_prefix":"71b67a4093fd6e16","entry":"computer_out_idx","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_filter_and_channel.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_filter_and_channel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"71b67a4093fd6e16"}},{"code_sha256_prefix":"587ea2d9b6e9d61d","entry":"create_batchnorm2d","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/layers.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"587ea2d9b6e9d61d"}},{"code_sha256_prefix":"a21ac6077cf08c3f","entry":"create_conv2d","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/layers.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a21ac6077cf08c3f"}},{"code_sha256_prefix":"b5f7cd56c2cc63e5","entry":"create_linear","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/layers.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/layers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b5f7cd56c2cc63e5"}},{"code_sha256_prefix":"d8ae353c6503316a","entry":"prune","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_resnet_by_depth.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_resnet_by_depth.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d8ae353c6503316a"}},{"code_sha256_prefix":"8400f740e15a35f2","entry":"prune_bottleneck","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_resnet_by_depth.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_resnet_by_depth.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8400f740e15a35f2"}},{"code_sha256_prefix":"549292cb3c24a501","entry":"prune_classifier","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_filter.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_filter.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"549292cb3c24a501"}},{"code_sha256_prefix":"eeea4d1c97d32bc2","entry":"prune_conv_bn_relu","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_channel.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_channel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eeea4d1c97d32bc2"}},{"code_sha256_prefix":"3b1a7ef75bcdd7fd","entry":"prune_conv_bn_relu","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_filter.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_filter.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b1a7ef75bcdd7fd"}},{"code_sha256_prefix":"9936b8b0a479ae6f","entry":"prune_conv_bn_relu","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_filter_and_channel.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_filter_and_channel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9936b8b0a479ae6f"}},{"code_sha256_prefix":"af5704b466281ae7","entry":"prune_features","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_channel.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_channel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af5704b466281ae7"}},{"code_sha256_prefix":"ac785d648e672e56","entry":"prune_features","repo":"ZJCV/SSL","repo_kind":"listed","path":"sslearning/prune/prune_vggnet_by_filter.py","file_url":"https://github.com/ZJCV/SSL/blob/HEAD/sslearning/prune/prune_vggnet_by_filter.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac785d648e672e56"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}