{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-neural-network-compression-with-single","title":"Deep Neural Network Compression with Single and Multiple Level Quantization","arxiv_id":"1803.03289","date":"2018-03-06","proceeding":null,"authors":["Yuhui Xu","Yongzhuang Wang","Aojun Zhou","Weiyao Lin","Hongkai Xiong"],"abstract":"Network quantization is an effective solution to compress deep neural\nnetworks for practical usage. Existing network quantization methods cannot\nsufficiently exploit the depth information to generate low-bit compressed\nnetwork. In this paper, we propose two novel network quantization approaches,\nsingle-level network quantization (SLQ) for high-bit quantization and\nmulti-level network quantization (MLQ) for extremely low-bit quantization\n(ternary).We are the first to consider the network quantization from both width\nand depth level. In the width level, parameters are divided into two parts: one\nfor quantization and the other for re-training to eliminate the quantization\nloss. SLQ leverages the distribution of the parameters to improve the width\nlevel. In the depth level, we introduce incremental layer compensation to\nquantize layers iteratively which decreases the quantization loss in each\niteration. The proposed approaches are validated with extensive experiments\nbased on the state-of-the-art neural networks including AlexNet, VGG-16,\nGoogleNet and ResNet-18. Both SLQ and MLQ achieve impressive results.","url_abs":"http://arxiv.org/abs/1803.03289v2","url_pdf":"http://arxiv.org/pdf/1803.03289v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-neural-network-compression-with-single","repo_url":"https://github.com/yuhuixu1993/SLQ","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"neural-network-compression","task_name":"Neural Network Compression"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1803.03289","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}