{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mixed-precision-neural-network-quantization","title":"Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance","arxiv_id":"2203.08368","date":"2022-03-16","proceeding":null,"authors":["Chen Tang","Kai Ouyang","Zhi Wang","Yifei Zhu","YaoWei Wang","Wen Ji","Wenwu Zhu"],"abstract":"The exponentially large discrete search space in mixed-precision quantization (MPQ) makes it hard to determine the optimal bit-width for each layer. Previous works usually resort to iterative search methods on the training set, which consume hundreds or even thousands of GPU-hours. In this study, we reveal that some unique learnable parameters in quantization, namely the scale factors in the quantizer, can serve as importance indicators of a layer, reflecting the contribution of that layer to the final accuracy at certain bit-widths. These importance indicators naturally perceive the numerical transformation during quantization-aware training, which can precisely provide quantization sensitivity metrics of layers. However, a deep network always contains hundreds of such indicators, and training them one by one would lead to an excessive time cost. To overcome this issue, we propose a joint training scheme that can obtain all indicators at once. It considerably speeds up the indicators training process by parallelizing the original sequential training processes. With these learned importance indicators, we formulate the MPQ search problem as a one-time integer linear programming (ILP) problem. That avoids the iterative search and significantly reduces search time without limiting the bit-width search space. For example, MPQ search on ResNet18 with our indicators takes only 0.06 s, which improves time efficiency exponentially compared to iterative search methods. Also, extensive experiments show our approach can achieve SOTA accuracy on ImageNet for far-ranging models with various constraints (e.g., BitOps, compress rate). Code is available on https://github.com/1hunters/LIMPQ.","url_abs":"https://arxiv.org/abs/2203.08368v5","url_pdf":"https://arxiv.org/pdf/2203.08368v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mixed-precision-neural-network-quantization","repo_url":"https://github.com/1hunters/LIMPQ","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.08368","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.08368"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/1hunters/LIMPQ","reach":null}],"summary":{"ran":2,"ran_honours":3,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"19b82b339c16e729","entry":"LsqQuan","repo":"1hunters/LIMPQ","repo_kind":"official","path":"quantization_training/quan/quantizer/lsq.py","file_url":"https://github.com/1hunters/LIMPQ/blob/HEAD/quantization_training/quan/quantizer/lsq.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19b82b339c16e729"}},{"code_sha256_prefix":"c72e6780cb69622d","entry":"Quantizer","repo":"1hunters/LIMPQ","repo_kind":"official","path":"quantization_training/quan/quantizer/lsq.py","file_url":"https://github.com/1hunters/LIMPQ/blob/HEAD/quantization_training/quan/quantizer/lsq.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c72e6780cb69622d"}},{"code_sha256_prefix":"47babd16aec6e5d9","entry":"compute_thd","repo":"1hunters/LIMPQ","repo_kind":"official","path":"quantization_training/quan/quantizer/lsq.py","file_url":"https://github.com/1hunters/LIMPQ/blob/HEAD/quantization_training/quan/quantizer/lsq.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47babd16aec6e5d9"}},{"code_sha256_prefix":"a488bae52f1bfd5d","entry":"grad_scale","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"a488bae52f1bfd5d"}},{"code_sha256_prefix":"ed0c43d8f1ea3ee2","entry":"quant_operator","repo":"1hunters/LIMPQ","repo_kind":"official","path":"quantization_training/quan/quantizer/lsq.py","file_url":"https://github.com/1hunters/LIMPQ/blob/HEAD/quantization_training/quan/quantizer/lsq.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ed0c43d8f1ea3ee2"}},{"code_sha256_prefix":"ed1a69e092272dba","entry":"round_pass","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"ed1a69e092272dba"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}