{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/network-binarization-via-contrastive-learning","title":"Network Binarization via Contrastive Learning","arxiv_id":"2207.02970","date":"2022-07-06","proceeding":null,"authors":["Yuzhang Shang","Dan Xu","Ziliang Zong","Liqiang Nie","Yan Yan"],"abstract":"Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit. However, there is still a huge performance gap between Binary Neural Networks (BNNs) and their full-precision (FP) counterparts. As the quantization error caused by weights binarization has been reduced in earlier works, the activations binarization becomes the major obstacle for further improvement of the accuracy. BNN characterises a unique and interesting structure, where the binary and latent FP activations exist in the same forward pass (i.e., $\\text{Binarize}(\\mathbf{a}_F) = \\mathbf{a}_B$). To mitigate the information degradation caused by the binarization operation from FP to binary activations, we establish a novel contrastive learning framework while training BNNs through the lens of Mutual Information (MI) maximization. MI is introduced as the metric to measure the information shared between binary and FP activations, which assists binarization with contrastive learning. Specifically, the representation ability of the BNNs is greatly strengthened via pulling the positive pairs with binary and FP activations from the same input samples, as well as pushing negative pairs from different samples (the number of negative pairs can be exponentially large). This benefits the downstream tasks, not only classification but also segmentation and depth estimation, etc. The experimental results show that our method can be implemented as a pile-up module on existing state-of-the-art binarization methods and can remarkably improve the performance over them on CIFAR-10/100 and ImageNet, in addition to the great generalization ability on NYUD-v2.","url_abs":"https://arxiv.org/abs/2207.02970v3","url_pdf":"https://arxiv.org/pdf/2207.02970v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"network-binarization-via-contrastive-learning","repo_url":"https://github.com/42Shawn/CMIM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"binarization","task_name":"Binarization"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.02970","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.02970"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/42Shawn/CMIM","reach":null}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f9f0276f9881c6ce","entry":"ContrastLoss","repo":"42Shawn/CMIM","repo_kind":"official","path":"CMIM/cifar100/contrastive_alignment/criterion.py","file_url":"https://github.com/42Shawn/CMIM/blob/HEAD/CMIM/cifar100/contrastive_alignment/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9f0276f9881c6ce"}},{"code_sha256_prefix":"a10e610c2b6a4903","entry":"ContrastMemory","repo":"42Shawn/CMIM","repo_kind":"official","path":"CMIM/cifar100/contrastive_alignment/criterion.py","file_url":"https://github.com/42Shawn/CMIM/blob/HEAD/CMIM/cifar100/contrastive_alignment/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a10e610c2b6a4903"}},{"code_sha256_prefix":"6fb273c6076eb82c","entry":"Embed","repo":"42Shawn/CMIM","repo_kind":"official","path":"CMIM/cifar100/contrastive_alignment/criterion.py","file_url":"https://github.com/42Shawn/CMIM/blob/HEAD/CMIM/cifar100/contrastive_alignment/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6fb273c6076eb82c"}},{"code_sha256_prefix":"3761ae6d7aa4a39a","entry":"contrastive_alignment","repo":"42Shawn/CMIM","repo_kind":"official","path":"CMIM/cifar100/contrastive_alignment/criterion.py","file_url":"https://github.com/42Shawn/CMIM/blob/HEAD/CMIM/cifar100/contrastive_alignment/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3761ae6d7aa4a39a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}