{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-network-surgery-for-efficient-dnns","title":"Dynamic Network Surgery for Efficient DNNs","arxiv_id":"1608.04493","date":"2016-08-16","proceeding":"NeurIPS 2016 12","authors":["Yiwen Guo","Anbang Yao","Yurong Chen"],"abstract":"Deep learning has become a ubiquitous technology to improve machine\nintelligence. However, most of the existing deep models are structurally very\ncomplex, making them difficult to be deployed on the mobile platforms with\nlimited computational power. In this paper, we propose a novel network\ncompression method called dynamic network surgery, which can remarkably reduce\nthe network complexity by making on-the-fly connection pruning. Unlike the\nprevious methods which accomplish this task in a greedy way, we properly\nincorporate connection splicing into the whole process to avoid incorrect\npruning and make it as a continual network maintenance. The effectiveness of\nour method is proved with experiments. Without any accuracy loss, our method\ncan efficiently compress the number of parameters in LeNet-5 and AlexNet by a\nfactor of $\\bm{108}\\times$ and $\\bm{17.7}\\times$ respectively, proving that it\noutperforms the recent pruning method by considerable margins. Code and some\nmodels are available at https://github.com/yiwenguo/Dynamic-Network-Surgery.","url_abs":"http://arxiv.org/abs/1608.04493v2","url_pdf":"http://arxiv.org/pdf/1608.04493v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-network-surgery-for-efficient-dnns","repo_url":"https://github.com/yiwenguo/Dynamic-Network-Surgery","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"dynamic-network-surgery-for-efficient-dnns","repo_url":"https://github.com/Subhranil24/NeuralPruning-ReImplementingDynamicNetworkSurgery","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"dynamic-network-surgery-for-efficient-dnns","repo_url":"https://github.com/maheshkaran/Nervana-Distiller","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1608.04493","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}