{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/regularizing-activation-distribution-for","title":"Regularizing Activation Distribution for Training Binarized Deep Networks","arxiv_id":"1904.02823","date":"2019-04-04","proceeding":"CVPR 2019 6","authors":["Ruizhou Ding","Ting-Wu Chin","Zeye Liu","Diana Marculescu"],"abstract":"Binarized Neural Networks (BNNs) can significantly reduce the inference\nlatency and energy consumption in resource-constrained devices due to their\npure-logical computation and fewer memory accesses. However, training BNNs is\ndifficult since the activation flow encounters degeneration, saturation, and\ngradient mismatch problems. Prior work alleviates these issues by increasing\nactivation bits and adding floating-point scaling factors, thereby sacrificing\nBNN's energy efficiency. In this paper, we propose to use distribution loss to\nexplicitly regularize the activation flow, and develop a framework to\nsystematically formulate the loss. Our experiments show that the distribution\nloss can consistently improve the accuracy of BNNs without losing their energy\nbenefits. Moreover, equipped with the proposed regularization, BNN training is\nshown to be robust to the selection of hyper-parameters including optimizer and\nlearning rate.","url_abs":"http://arxiv.org/abs/1904.02823v1","url_pdf":"http://arxiv.org/pdf/1904.02823v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"regularizing-activation-distribution-for","repo_url":"https://github.com/ruizhoud/DistributionLoss","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1904.02823","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}