{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hybrid-binary-networks-optimizing-for","title":"Hybrid Binary Networks: Optimizing for Accuracy, Efficiency and Memory","arxiv_id":"1804.03867","date":"2018-04-11","proceeding":null,"authors":["Ameya Prabhu","Vishal Batchu","Rohit Gajawada","Sri Aurobindo Munagala","Anoop Namboodiri"],"abstract":"Binarization is an extreme network compression approach that provides large\ncomputational speedups along with energy and memory savings, albeit at\nsignificant accuracy costs. We investigate the question of where to binarize\ninputs at layer-level granularity and show that selectively binarizing the\ninputs to specific layers in the network could lead to significant improvements\nin accuracy while preserving most of the advantages of binarization. We analyze\nthe binarization tradeoff using a metric that jointly models the input\nbinarization-error and computational cost and introduce an efficient algorithm\nto select layers whose inputs are to be binarized. Practical guidelines based\non insights obtained from applying the algorithm to a variety of models are\ndiscussed. Experiments on Imagenet dataset using AlexNet and ResNet-18 models\nshow 3-4% improvements in accuracy over fully binarized networks with minimal\nimpact on compression and computational speed. The improvements are even more\nsubstantial on sketch datasets like TU-Berlin, where we match state-of-the-art\naccuracy as well, getting over 8% increase in accuracies. We further show that\nour approach can be applied in tandem with other forms of compression that deal\nwith individual layers or overall model compression (e.g., SqueezeNets). Unlike\nprevious quantization approaches, we are able to binarize the weights in the\nlast layers of a network, which often have a large number of parameters,\nresulting in significant improvement in accuracy over fully binarized models.","url_abs":"http://arxiv.org/abs/1804.03867v1","url_pdf":"http://arxiv.org/pdf/1804.03867v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hybrid-binary-networks-optimizing-for","repo_url":"https://github.com/erilyth/HybridBinaryNetworks-WACV18","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"binarization","task_name":"Binarization"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}