{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/finding-the-task-optimal-low-bit-sub","title":"Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks","arxiv_id":"2112.15139","date":"2021-12-30","proceeding":null,"authors":["Runpei Dong","Zhanhong Tan","Mengdi Wu","Linfeng Zhang","Kaisheng Ma"],"abstract":"Quantized neural networks typically require smaller memory footprints and lower computation complexity, which is crucial for efficient deployment. However, quantization inevitably leads to a distribution divergence from the original network, which generally degrades the performance. To tackle this issue, massive efforts have been made, but most existing approaches lack statistical considerations and depend on several manual configurations. In this paper, we present an adaptive-mapping quantization method to learn an optimal latent sub-distribution that is inherent within models and smoothly approximated with a concrete Gaussian Mixture (GM). In particular, the network weights are projected in compliance with the GM-approximated sub-distribution. This sub-distribution evolves along with the weight update in a co-tuning schema guided by the direct task-objective optimization. Sufficient experiments on image classification and object detection over various modern architectures demonstrate the effectiveness, generalization property, and transferability of the proposed method. Besides, an efficient deployment flow for the mobile CPU is developed, achieving up to 7.46$\\times$ inference acceleration on an octa-core ARM CPU. Our codes have been publicly released at \\url{https://github.com/RunpeiDong/DGMS}.","url_abs":"https://arxiv.org/abs/2112.15139v4","url_pdf":"https://arxiv.org/pdf/2112.15139v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"finding-the-task-optimal-low-bit-sub","repo_url":"https://github.com/RunpeiDong/DGMS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.15139","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.15139"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/RunpeiDong/DGMS","reach":null}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6c5f8944444c02ec","entry":"DGMSConv","repo":"RunpeiDong/DGMS","repo_kind":"official","path":"modeling/DGMS/DGMSConv.py","file_url":"https://github.com/RunpeiDong/DGMS/blob/HEAD/modeling/DGMS/DGMSConv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c5f8944444c02ec"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}