{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recovering-from-random-pruning-on-the","title":"Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networks","arxiv_id":"1801.10447","date":"2018-01-31","proceeding":null,"authors":["Deepak Mittal","Shweta Bhardwaj","Mitesh M. Khapra","Balaraman Ravindran"],"abstract":"Recently there has been a lot of work on pruning filters from deep\nconvolutional neural networks (CNNs) with the intention of reducing\ncomputations. The key idea is to rank the filters based on a certain criterion\n(say, $l_1$-norm, average percentage of zeros, etc) and retain only the top\nranked filters. Once the low scoring filters are pruned away the remainder of\nthe network is fine tuned and is shown to give performance comparable to the\noriginal unpruned network. In this work, we report experiments which suggest\nthat the comparable performance of the pruned network is not due to the\nspecific criterion chosen but due to the inherent plasticity of deep neural\nnetworks which allows them to recover from the loss of pruned filters once the\nrest of the filters are fine-tuned. Specifically, we show counter-intuitive\nresults wherein by randomly pruning 25-50\\% filters from deep CNNs we are able\nto obtain the same performance as obtained by using state of the art pruning\nmethods. We empirically validate our claims by doing an exhaustive evaluation\nwith VGG-16 and ResNet-50. Further, we also evaluate a real world scenario\nwhere a CNN trained on all 1000 ImageNet classes needs to be tested on only a\nsmall set of classes at test time (say, only animals). We create a new\nbenchmark dataset from ImageNet to evaluate such class specific pruning and\nshow that even here a random pruning strategy gives close to state of the art\nperformance. Lastly, unlike existing approaches which mainly focus on the task\nof image classification, in this work we also report results on object\ndetection. We show that using a simple random pruning strategy we can achieve\nsignificant speed up in object detection (74$\\%$ improvement in fps) while\nretaining the same accuracy as that of the original Faster RCNN model.","url_abs":"http://arxiv.org/abs/1801.10447v1","url_pdf":"http://arxiv.org/pdf/1801.10447v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recovering-from-random-pruning-on-the","repo_url":"https://github.com/shwetabhardwaj44/RecoveringFrom_RandomPruning_WACV2018","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1801.10447","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}