{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-runtime-feature-map-pruning","title":"Dynamic Runtime Feature Map Pruning","arxiv_id":"1812.09922","date":"2018-12-24","proceeding":null,"authors":["Tailin Liang","Lei Wang","Shaobo Shi","John Glossner"],"abstract":"High bandwidth requirements are an obstacle for accelerating the training and\ninference of deep neural networks. Most previous research focuses on reducing\nthe size of kernel maps for inference. We analyze parameter sparsity of six\npopular convolutional neural networks - AlexNet, MobileNet, ResNet-50,\nSqueezeNet, TinyNet, and VGG16. Of the networks considered, those using ReLU\n(AlexNet, SqueezeNet, VGG16) contain a high percentage of 0-valued parameters\nand can be statically pruned. Networks with Non-ReLU activation functions in\nsome cases may not contain any 0-valued parameters (ResNet-50, TinyNet). We\nalso investigate runtime feature map usage and find that input feature maps\ncomprise the majority of bandwidth requirements when depth-wise convolution and\npoint-wise convolutions used. We introduce dynamic runtime pruning of feature\nmaps and show that 10% of dynamic feature map execution can be removed without\nloss of accuracy. We then extend dynamic pruning to allow for values within an\nepsilon of zero and show a further 5% reduction of feature map loading with a\n1% loss of accuracy in top-1.","url_abs":"http://arxiv.org/abs/1812.09922v2","url_pdf":"http://arxiv.org/pdf/1812.09922v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-runtime-feature-map-pruning","repo_url":"https://github.com/liangtailin/darknet-modified","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"fire-module","method_name":"Fire Module"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"squeezenet","method_name":"SqueezeNet"},{"method_slug":"xavier-initialization","method_name":"Xavier Initialization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}