{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-investigation-into-the-stochasticity-of","title":"An Investigation into the Stochasticity of Batch Whitening","arxiv_id":"2003.12327","date":"2020-03-27","proceeding":"CVPR 2020 6","authors":["Lei Huang","Lei Zhao","Yi Zhou","Fan Zhu","Li Liu","Ling Shao"],"abstract":"Batch Normalization (BN) is extensively employed in various network architectures by performing standardization within mini-batches. A full understanding of the process has been a central target in the deep learning communities. Unlike existing works, which usually only analyze the standardization operation, this paper investigates the more general Batch Whitening (BW). Our work originates from the observation that while various whitening transformations equivalently improve the conditioning, they show significantly different behaviors in discriminative scenarios and training Generative Adversarial Networks (GANs). We attribute this phenomenon to the stochasticity that BW introduces. We quantitatively investigate the stochasticity of different whitening transformations and show that it correlates well with the optimization behaviors during training. We also investigate how stochasticity relates to the estimation of population statistics during inference. Based on our analysis, we provide a framework for designing and comparing BW algorithms in different scenarios. Our proposed BW algorithm improves the residual networks by a significant margin on ImageNet classification. Besides, we show that the stochasticity of BW can improve the GAN's performance with, however, the sacrifice of the training stability.","url_abs":"https://arxiv.org/abs/2003.12327v1","url_pdf":"https://arxiv.org/pdf/2003.12327v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-investigation-into-the-stochasticity-of","repo_url":"https://github.com/huangleiBuaa/StochasticityBW","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.12327","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.12327"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huangleiBuaa/StochasticityBW","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"huangleiBuaa/StochasticityBW","repo_kind":"official","path":"SBW_Classification_PyTorch/ImageNet/models/resnet.py","file_url":"https://github.com/huangleiBuaa/StochasticityBW/blob/HEAD/SBW_Classification_PyTorch/ImageNet/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"a8ef2238d374d6cd","entry":"accuracy","repo":"huangleiBuaa/StochasticityBW","repo_kind":"official","path":"SBW_Classification_PyTorch/ImageNet/imagenet.py","file_url":"https://github.com/huangleiBuaa/StochasticityBW/blob/HEAD/SBW_Classification_PyTorch/ImageNet/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"a8ef2238d374d6cd"}},{"code_sha256_prefix":"cacaa2915e934d81","entry":"to_img","repo":"huangleiBuaa/StochasticityBW","repo_kind":"official","path":"SBW_Classification_PyTorch/Mnist/mnist.py","file_url":"https://github.com/huangleiBuaa/StochasticityBW/blob/HEAD/SBW_Classification_PyTorch/Mnist/mnist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cacaa2915e934d81"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}