{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/estimator-meets-equilibrium-perspective-a","title":"Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training","arxiv_id":"2308.06689","date":"2023-08-13","proceeding":"ICCV 2023 1","authors":["Xiao-Ming Wu","Dian Zheng","Zuhao Liu","Wei-Shi Zheng"],"abstract":"Binarization of neural networks is a dominant paradigm in neural networks compression. The pioneering work BinaryConnect uses Straight Through Estimator (STE) to mimic the gradients of the sign function, but it also causes the crucial inconsistency problem. Most of the previous methods design different estimators instead of STE to mitigate it. However, they ignore the fact that when reducing the estimating error, the gradient stability will decrease concomitantly. These highly divergent gradients will harm the model training and increase the risk of gradient vanishing and gradient exploding. To fully take the gradient stability into consideration, we present a new perspective to the BNNs training, regarding it as the equilibrium between the estimating error and the gradient stability. In this view, we firstly design two indicators to quantitatively demonstrate the equilibrium phenomenon. In addition, in order to balance the estimating error and the gradient stability well, we revise the original straight through estimator and propose a power function based estimator, Rectified Straight Through Estimator (ReSTE for short). Comparing to other estimators, ReSTE is rational and capable of flexibly balancing the estimating error with the gradient stability. Extensive experiments on CIFAR-10 and ImageNet datasets show that ReSTE has excellent performance and surpasses the state-of-the-art methods without any auxiliary modules or losses.","url_abs":"https://arxiv.org/abs/2308.06689v2","url_pdf":"https://arxiv.org/pdf/2308.06689v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"estimator-meets-equilibrium-perspective-a","repo_url":"https://github.com/dravenalg/reste","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"binarization","task_name":"Binarization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.06689","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.06689"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dravenalg/reste","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/DravenALG/ReSTE","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"09e796c558171dac","entry":"Binary_ReSTE","repo":"DravenALG/ReSTE","repo_kind":"official","path":"model/binary_module/binarized_modules.py","file_url":"https://github.com/DravenALG/ReSTE/blob/HEAD/model/binary_module/binarized_modules.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09e796c558171dac"}},{"code_sha256_prefix":"116d1448f8e9a8bd","entry":"accuracy","repo":"dravenalg/reste","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"116d1448f8e9a8bd"}},{"code_sha256_prefix":"157ec370c18f8930","entry":"add_module_fromdict","repo":"dravenalg/reste","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"157ec370c18f8930"}},{"code_sha256_prefix":"09cf7bf4ad177244","entry":"approximate_function","repo":"DravenALG/ReSTE","repo_kind":"official","path":"model/binary_module/binarized_modules.py","file_url":"https://github.com/DravenALG/ReSTE/blob/HEAD/model/binary_module/binarized_modules.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09cf7bf4ad177244"}},{"code_sha256_prefix":"19fd0375add14cbc","entry":"delete_module_fromdict","repo":"dravenalg/reste","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"19fd0375add14cbc"}},{"code_sha256_prefix":"e0bd0b127c31b7df","entry":"load","repo":"dravenalg/reste","repo_kind":"official","path":"dataset/load_data.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/dataset/load_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e0bd0b127c31b7df"}},{"code_sha256_prefix":"7386d10841a4a333","entry":"resnet18","repo":"dravenalg/reste","repo_kind":"official","path":"model/models_imagenet/resnet18.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/model/models_imagenet/resnet18.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7386d10841a4a333"}},{"code_sha256_prefix":"e3761947f732d1fe","entry":"resnet18_1w1a","repo":"dravenalg/reste","repo_kind":"official","path":"model/models_imagenet/resnet18.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/model/models_imagenet/resnet18.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3761947f732d1fe"}},{"code_sha256_prefix":"7ee85694bd6b7162","entry":"resnet34_1w1a","repo":"dravenalg/reste","repo_kind":"official","path":"model/models_imagenet/resnet34.py","file_url":"https://github.com/dravenalg/reste/blob/HEAD/model/models_imagenet/resnet34.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ee85694bd6b7162"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}