{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-systematic-dnn-weight-pruning-framework","title":"A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers","arxiv_id":"1804.03294","date":"2018-04-10","proceeding":"ECCV 2018 9","authors":["Tianyun Zhang","Shaokai Ye","Kaiqi Zhang","Jian Tang","Wujie Wen","Makan Fardad","Yanzhi Wang"],"abstract":"Weight pruning methods for deep neural networks (DNNs) have been investigated\nrecently, but prior work in this area is mainly heuristic, iterative pruning,\nthereby lacking guarantees on the weight reduction ratio and convergence time.\nTo mitigate these limitations, we present a systematic weight pruning framework\nof DNNs using the alternating direction method of multipliers (ADMM). We first\nformulate the weight pruning problem of DNNs as a nonconvex optimization\nproblem with combinatorial constraints specifying the sparsity requirements,\nand then adopt the ADMM framework for systematic weight pruning. By using ADMM,\nthe original nonconvex optimization problem is decomposed into two subproblems\nthat are solved iteratively. One of these subproblems can be solved using\nstochastic gradient descent, the other can be solved analytically. Besides, our\nmethod achieves a fast convergence rate.\n  The weight pruning results are very promising and consistently outperform the\nprior work. On the LeNet-5 model for the MNIST data set, we achieve 71.2 times\nweight reduction without accuracy loss. On the AlexNet model for the ImageNet\ndata set, we achieve 21 times weight reduction without accuracy loss. When we\nfocus on the convolutional layer pruning for computation reductions, we can\nreduce the total computation by five times compared with the prior work\n(achieving a total of 13.4 times weight reduction in convolutional layers). Our\nmodels and codes are released at https://github.com/KaiqiZhang/admm-pruning","url_abs":"http://arxiv.org/abs/1804.03294v3","url_pdf":"http://arxiv.org/pdf/1804.03294v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-systematic-dnn-weight-pruning-framework","repo_url":"https://github.com/KaiqiZhang/admm-pruning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"a-systematic-dnn-weight-pruning-framework","repo_url":"https://github.com/KaiqiZhang/caffe-admm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"a-systematic-dnn-weight-pruning-framework","repo_url":"https://github.com/anonymous2020-source-code/wnfg-ssgcnet-admm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-systematic-dnn-weight-pruning-framework","repo_url":"https://github.com/luxx0235/ADMM-Pruning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"network-pruning","task_name":"Network Pruning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"admm","method_name":"ADMM"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1804.03294","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1804.03294"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KaiqiZhang/admm-pruning","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anonymous2020-source-code/wnfg-ssgcnet-admm","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/luxx0235/ADMM-Pruning","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/KaiqiZhang/caffe-admm","reach":{"status":"unanswered"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"d48616ed9328d731","entry":"admm_loss","repo":"anonymous2020-source-code/wnfg-ssgcnet-admm","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/anonymous2020-source-code/wnfg-ssgcnet-admm/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d48616ed9328d731"}},{"code_sha256_prefix":"53a03be5fbeab4da","entry":"initialize_Z_and_U","repo":"anonymous2020-source-code/wnfg-ssgcnet-admm","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/anonymous2020-source-code/wnfg-ssgcnet-admm/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"53a03be5fbeab4da"}},{"code_sha256_prefix":"49c167541a15b9c2","entry":"regularized_nll_loss","repo":"anonymous2020-source-code/wnfg-ssgcnet-admm","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/anonymous2020-source-code/wnfg-ssgcnet-admm/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49c167541a15b9c2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}