{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/incremental-network-quantization-towards","title":"Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights","arxiv_id":"1702.03044","date":"2017-02-10","proceeding":null,"authors":["Aojun Zhou","Anbang Yao","Yiwen Guo","Lin Xu","Yurong Chen"],"abstract":"This paper presents incremental network quantization (INQ), a novel method,\ntargeting to efficiently convert any pre-trained full-precision convolutional\nneural network (CNN) model into a low-precision version whose weights are\nconstrained to be either powers of two or zero. Unlike existing methods which\nare struggled in noticeable accuracy loss, our INQ has the potential to resolve\nthis issue, as benefiting from two innovations. On one hand, we introduce three\ninterdependent operations, namely weight partition, group-wise quantization and\nre-training. A well-proven measure is employed to divide the weights in each\nlayer of a pre-trained CNN model into two disjoint groups. The weights in the\nfirst group are responsible to form a low-precision base, thus they are\nquantized by a variable-length encoding method. The weights in the other group\nare responsible to compensate for the accuracy loss from the quantization, thus\nthey are the ones to be re-trained. On the other hand, these three operations\nare repeated on the latest re-trained group in an iterative manner until all\nthe weights are converted into low-precision ones, acting as an incremental\nnetwork quantization and accuracy enhancement procedure. Extensive experiments\non the ImageNet classification task using almost all known deep CNN\narchitectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the\nefficacy of the proposed method. Specifically, at 5-bit quantization, our\nmodels have improved accuracy than the 32-bit floating-point references. Taking\nResNet-18 as an example, we further show that our quantized models with 4-bit,\n3-bit and 2-bit ternary weights have improved or very similar accuracy against\nits 32-bit floating-point baseline. Besides, impressive results with the\ncombination of network pruning and INQ are also reported. The code is available\nat https://github.com/Zhouaojun/Incremental-Network-Quantization.","url_abs":"http://arxiv.org/abs/1702.03044v2","url_pdf":"http://arxiv.org/pdf/1702.03044v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"incremental-network-quantization-towards","repo_url":"https://github.com/Zhouaojun/Incremental-Network-Quantization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"incremental-network-quantization-towards","repo_url":"https://github.com/Mxbonn/INQ-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"incremental-network-quantization-towards","repo_url":"https://github.com/chuanraoCV/INQ-incremental-network-quantization-towards-lossless-CNNs-with-low-precision-weights","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"auxiliary-classifier","method_name":"Auxiliary Classifier"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"googlenet","method_name":"GoogLeNet"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"inception-module","method_name":"Inception Module"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1702.03044","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}