{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/group-whitening-balancing-learning-efficiency","title":"Group Whitening: Balancing Learning Efficiency and Representational Capacity","arxiv_id":"2009.13333","date":"2020-09-28","proceeding":"CVPR 2021 1","authors":["Lei Huang","Yi Zhou","Li Liu","Fan Zhu","Ling Shao"],"abstract":"Batch normalization (BN) is an important technique commonly incorporated into deep learning models to perform standardization within mini-batches. The merits of BN in improving a model's learning efficiency can be further amplified by applying whitening, while its drawbacks in estimating population statistics for inference can be avoided through group normalization (GN). This paper proposes group whitening (GW), which exploits the advantages of the whitening operation and avoids the disadvantages of normalization within mini-batches. In addition, we analyze the constraints imposed on features by normalization, and show how the batch size (group number) affects the performance of batch (group) normalized networks, from the perspective of model's representational capacity. This analysis provides theoretical guidance for applying GW in practice. Finally, we apply the proposed GW to ResNet and ResNeXt architectures and conduct experiments on the ImageNet and COCO benchmarks. Results show that GW consistently improves the performance of different architectures, with absolute gains of $1.02\\%$ $\\sim$ $1.49\\%$ in top-1 accuracy on ImageNet and $1.82\\%$ $\\sim$ $3.21\\%$ in bounding box AP on COCO.","url_abs":"https://arxiv.org/abs/2009.13333v4","url_pdf":"https://arxiv.org/pdf/2009.13333v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"group-whitening-balancing-learning-efficiency","repo_url":"https://github.com/huangleiBuaa/GroupWhitening","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"group-normalization","method_name":"Group Normalization"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"resnext","method_name":"ResNeXt"},{"method_slug":"resnext-block","method_name":"ResNeXt Block"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2009.13333","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2009.13333"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huangleiBuaa/GroupWhitening","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"summary":{"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"listed":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"huangleiBuaa/GroupWhitening","repo_kind":"listed","path":"classification/ImageNet/models/resnext.py","file_url":"https://github.com/huangleiBuaa/GroupWhitening/blob/HEAD/classification/ImageNet/models/resnext.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"huangleiBuaa/GroupWhitening","repo_kind":"listed","path":"classification/ImageNet/models/resnext.py","file_url":"https://github.com/huangleiBuaa/GroupWhitening/blob/HEAD/classification/ImageNet/models/resnext.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"huangleiBuaa/GroupWhitening","repo_kind":"listed","path":"classification/ImageNet/models/resnet.py","file_url":"https://github.com/huangleiBuaa/GroupWhitening/blob/HEAD/classification/ImageNet/models/resnet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"a8ef2238d374d6cd","entry":"accuracy","repo":"huangleiBuaa/GroupWhitening","repo_kind":"listed","path":"classification/ImageNet/imagenet.py","file_url":"https://github.com/huangleiBuaa/GroupWhitening/blob/HEAD/classification/ImageNet/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a8ef2238d374d6cd"}},{"code_sha256_prefix":"cacaa2915e934d81","entry":"to_img","repo":"huangleiBuaa/GroupWhitening","repo_kind":"listed","path":"classification/Mnist/mnist.py","file_url":"https://github.com/huangleiBuaa/GroupWhitening/blob/HEAD/classification/Mnist/mnist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cacaa2915e934d81"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}