{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/190503696","title":"HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision","arxiv_id":"1905.03696","date":"2019-04-29","proceeding":"ICCV 2019 10","authors":["Zhen Dong","Zhewei Yao","Amir Gholami","Michael Mahoney","Kurt Keutzer"],"abstract":"Model size and inference speed/power have become a major challenge in the\ndeployment of Neural Networks for many applications. A promising approach to\naddress these problems is quantization. However, uniformly quantizing a model\nto ultra low precision leads to significant accuracy degradation. A novel\nsolution for this is to use mixed-precision quantization, as some parts of the\nnetwork may allow lower precision as compared to other layers. However, there\nis no systematic way to determine the precision of different layers. A brute\nforce approach is not feasible for deep networks, as the search space for\nmixed-precision is exponential in the number of layers. Another challenge is a\nsimilar factorial complexity for determining block-wise fine-tuning order when\nquantizing the model to a target precision. Here, we introduce Hessian AWare\nQuantization (HAWQ), a novel second-order quantization method to address these\nproblems. HAWQ allows for the automatic selection of the relative quantization\nprecision of each layer, based on the layer's Hessian spectrum. Moreover, HAWQ\nprovides a deterministic fine-tuning order for quantizing layers, based on\nsecond-order information. We show the results of our method on Cifar-10 using\nResNet20, and on ImageNet using Inception-V3, ResNet50 and SqueezeNext models.\nComparing HAWQ with state-of-the-art shows that we can achieve similar/better\naccuracy with $8\\times$ activation compression ratio on ResNet20, as compared\nto DNAS~\\cite{wu2018mixed}, and up to $1\\%$ higher accuracy with up to $14\\%$\nsmaller models on ResNet50 and Inception-V3, compared to recently proposed\nmethods of RVQuant~\\cite{park2018value} and HAQ~\\cite{wang2018haq}.\nFurthermore, we show that we can quantize SqueezeNext to just 1MB model size\nwhile achieving above $68\\%$ top1 accuracy on ImageNet.","url_abs":"http://arxiv.org/abs/1905.03696v1","url_pdf":"http://arxiv.org/pdf/1905.03696v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"190503696","repo_url":"https://github.com/zhen-dong/hawq","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spatially-separable-convolution","method_name":"Spatially Separable Convolution"},{"method_slug":"squeezenext","method_name":"SqueezeNeXt"},{"method_slug":"squeezenext-block","method_name":"SqueezeNeXt Block"},{"method_slug":"xavier-initialization","method_name":"Xavier Initialization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1905.03696","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.03696"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhen-dong/hawq","reach":null}],"summary":{"ran_fixture":1,"unverified":2},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"131a82fd65128218","entry":"accuracy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"131a82fd65128218"}},{"code_sha256_prefix":"f48fafae2a9b58b2","entry":"loss_kd","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f48fafae2a9b58b2"}},{"code_sha256_prefix":"91153a2b2d1f9895","entry":"validate","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"91153a2b2d1f9895"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}