{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recu-reviving-the-dead-weights-in-binary","title":"ReCU: Reviving the Dead Weights in Binary Neural Networks","arxiv_id":"2103.12369","date":"2021-03-23","proceeding":"ICCV 2021 10","authors":["Zihan Xu","Mingbao Lin","Jianzhuang Liu","Jie Chen","Ling Shao","Yue Gao","Yonghong Tian","Rongrong Ji"],"abstract":"Binary neural networks (BNNs) have received increasing attention due to their superior reductions of computation and memory. Most existing works focus on either lessening the quantization error by minimizing the gap between the full-precision weights and their binarization or designing a gradient approximation to mitigate the gradient mismatch, while leaving the \"dead weights\" untouched. This leads to slow convergence when training BNNs. In this paper, for the first time, we explore the influence of \"dead weights\" which refer to a group of weights that are barely updated during the training of BNNs, and then introduce rectified clamp unit (ReCU) to revive the \"dead weights\" for updating. We prove that reviving the \"dead weights\" by ReCU can result in a smaller quantization error. Besides, we also take into account the information entropy of the weights, and then mathematically analyze why the weight standardization can benefit BNNs. We demonstrate the inherent contradiction between minimizing the quantization error and maximizing the information entropy, and then propose an adaptive exponential scheduler to identify the range of the \"dead weights\". By considering the \"dead weights\", our method offers not only faster BNN training, but also state-of-the-art performance on CIFAR-10 and ImageNet, compared with recent methods. Code can be available at https://github.com/z-hXu/ReCU.","url_abs":"https://arxiv.org/abs/2103.12369v2","url_pdf":"https://arxiv.org/pdf/2103.12369v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recu-reviving-the-dead-weights-in-binary","repo_url":"https://github.com/z-hXu/ReCU","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"paper_slug":"recu-reviving-the-dead-weights-in-binary","repo_url":"https://github.com/stevetsui/rbonn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"recu-reviving-the-dead-weights-in-binary","repo_url":"https://github.com/stevetsui/rebnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"binarization","task_name":"Binarization"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"weight-standardization","method_name":"Weight Standardization"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.12369","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.12369"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stevetsui/rebnn","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/z-hXu/ReCU","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stevetsui/rbonn","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"listed":{"samples":2,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"688635f52b25f29c","entry":"Binarization","repo":"stevetsui/rebnn","repo_kind":"listed","path":"2_step2_rebnn/rebnn.py","file_url":"https://github.com/stevetsui/rebnn/blob/HEAD/2_step2_rebnn/rebnn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"688635f52b25f29c"}},{"code_sha256_prefix":"5111a3243a80feb0","entry":"round_pass","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"5111a3243a80feb0"}},{"code_sha256_prefix":"7d50863cee4ef06c","entry":"RBOConv","repo":"stevetsui/rbonn","repo_kind":"listed","path":"2_step2_rbonn/rbonn.py","file_url":"https://github.com/stevetsui/rbonn/blob/HEAD/2_step2_rbonn/rbonn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d50863cee4ef06c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}