{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-bits-unifying-quantization-and","title":"Bayesian Bits: Unifying Quantization and Pruning","arxiv_id":"2005.07093","date":"2020-05-14","proceeding":"NeurIPS 2020 12","authors":["Mart van Baalen","Christos Louizos","Markus Nagel","Rana Ali Amjad","Ying Wang","Tijmen Blankevoort","Max Welling"],"abstract":"We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition of the quantization operation, which sequentially considers doubling the bit width. At each new bit width, the residual error between the full precision value and the previously rounded value is quantized. We then decide whether or not to add this quantized residual error for a higher effective bit width and lower quantization noise. By starting with a power-of-two bit width, this decomposition will always produce hardware-friendly configurations, and through an additional 0-bit option, serves as a unified view of pruning and quantization. Bayesian Bits then introduces learnable stochastic gates, which collectively control the bit width of the given tensor. As a result, we can obtain low bit solutions by performing approximate inference over the gates, with prior distributions that encourage most of them to be switched off. We experimentally validate our proposed method on several benchmark datasets and show that we can learn pruned, mixed precision networks that provide a better trade-off between accuracy and efficiency than their static bit width equivalents.","url_abs":"https://arxiv.org/abs/2005.07093v3","url_pdf":"https://arxiv.org/pdf/2005.07093v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-bits-unifying-quantization-and","repo_url":"https://github.com/Qualcomm-AI-research/BayesianBits","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.07093","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.07093"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Qualcomm-AI-research/BayesianBits","reach":null}],"summary":{"ran_draft_wrong":1,"ran_violates":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"a1020b85c9b2758a","entry":"get_conv_args","repo":"Qualcomm-AI-research/BayesianBits","repo_kind":"listed","path":"anonymized_compression_package/quantization/autoquant_utils.py","file_url":"https://github.com/Qualcomm-AI-research/BayesianBits/blob/HEAD/anonymized_compression_package/quantization/autoquant_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause-Clear","inline_ok":false,"mcp_get_code":{"code_sha256":"a1020b85c9b2758a"}},{"code_sha256_prefix":"870c07d5c47a4c01","entry":"next_bn","repo":"Qualcomm-AI-research/BayesianBits","repo_kind":"listed","path":"anonymized_compression_package/quantization/autoquant_utils.py","file_url":"https://github.com/Qualcomm-AI-research/BayesianBits/blob/HEAD/anonymized_compression_package/quantization/autoquant_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause-Clear","inline_ok":false,"mcp_get_code":{"code_sha256":"870c07d5c47a4c01"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}