{"url":"/method/batch-normalization","slug":"batch-normalization","name":"Batch Normalization","full_name":"Batch Normalization","full_name_withheld":false,"description_markdown":"**Batch Normalization** aims to reduce internal covariate shift, and in doing so aims to accelerate the training of deep neural nets. It accomplishes this via a normalization step that fixes the means and variances of layer inputs. Batch Normalization also has a beneficial effect on the gradient flow through the network, by reducing the dependence of gradients on the scale of the parameters or of their initial values. This allows for use of much higher learning rates without the risk of divergence. Furthermore, batch normalization regularizes the model and reduces the need for [Dropout](https://paperswithcode.com/method/dropout).\r\n\r\nWe apply a batch normalization layer as follows for a minibatch $\\mathcal{B}$:\r\n\r\n$$ \\mu\\_{\\mathcal{B}} = \\frac{1}{m}\\sum^{m}\\_{i=1}x\\_{i} $$\r\n\r\n$$ \\sigma^{2}\\_{\\mathcal{B}} = \\frac{1}{m}\\sum^{m}\\_{i=1}\\left(x\\_{i}-\\mu\\_{\\mathcal{B}}\\right)^{2} $$\r\n\r\n$$ \\hat{x}\\_{i} = \\frac{x\\_{i} - \\mu\\_{\\mathcal{B}}}{\\sqrt{\\sigma^{2}\\_{\\mathcal{B}}+\\epsilon}} $$\r\n\r\n$$ y\\_{i} = \\gamma\\hat{x}\\_{i} + \\beta = \\text{BN}\\_{\\gamma, \\beta}\\left(x\\_{i}\\right) $$\r\n\r\nWhere $\\gamma$ and $\\beta$ are learnable parameters.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1502.03167v3","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/google/jax/blob/36f91261099b00194922bd93ed1286fe1c199724/jax/experimental/stax.py#L116","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Normalization","url":"/methods/category/normalization","pwc_aliases":[]}],"n_papers_tagged":6287,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Unsupervised Deep Clustering of MNIST with Triplet-Enhanced Convolutional Autoencoders","date":"2025-06-11","arxiv_id":"2506.10094","n_code_links":0,"syntology":null},{"paper":null,"title":"SpikeSMOKE: Spiking Neural Networks for Monocular 3D Object Detection with Cross-Scale Gated Coding","date":"2025-06-09","arxiv_id":"2506.07737","n_code_links":0,"syntology":null},{"paper":null,"title":"The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks","date":"2025-06-09","arxiv_id":"2506.08274","n_code_links":0,"syntology":null},{"paper":null,"title":"Reliable Critics: Monotonic Improvement and Convergence Guarantees for Reinforcement Learning","date":"2025-06-08","arxiv_id":"2506.07134","n_code_links":0,"syntology":null},{"paper":null,"title":"GS4: Generalizable Sparse Splatting Semantic SLAM","date":"2025-06-06","arxiv_id":"2506.06517","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep histological synthesis from mass spectrometry imaging for multimodal registration","date":"2025-06-05","arxiv_id":"2506.05441","n_code_links":0,"syntology":null},{"paper":null,"title":"U-NetMN and SegNetMN: Modified U-Net and SegNet models for bimodal SAR image segmentation","date":"2025-06-05","arxiv_id":"2506.05444","n_code_links":0,"syntology":null},{"paper":null,"title":"A Novel Deep Reinforcement Learning Method for Computation Offloading in Multi-User Mobile Edge Computing with Decentralization","date":"2025-06-03","arxiv_id":"2506.02458","n_code_links":0,"syntology":null},{"paper":"/paper/efficientfer-efficientnetv2-based-deep","title":"EfficientFER: EfficientNetv2 Based Deep Learning Approach for Facial Expression Recognition","date":"2025-06-02","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"DuAL-Net: A Hybrid Framework for Alzheimer's Disease Prediction from Whole-Genome Sequencing via Local SNP Windows and Global Annotations","date":"2025-05-31","arxiv_id":"2506.00673","n_code_links":0,"syntology":null},{"paper":null,"title":"Black-box Adversarial Attacks on CNN-based SLAM Algorithms","date":"2025-05-30","arxiv_id":"2505.24654","n_code_links":0,"syntology":null},{"paper":null,"title":"QGAN-based data augmentation for hybrid quantum-classical neural networks","date":"2025-05-30","arxiv_id":"2505.24780","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Learning-Based Breast Cancer Detection in Mammography: A Multi-Center Validation Study in Thai Population","date":"2025-05-29","arxiv_id":"2506.03177","n_code_links":0,"syntology":null},{"paper":null,"title":"Low-Complexity Transform Adjustments For Video Coding","date":"2025-05-29","arxiv_id":"2505.23618","n_code_links":0,"syntology":null},{"paper":null,"title":"Calibrated Value-Aware Model Learning with Stochastic Environment Models","date":"2025-05-28","arxiv_id":"2505.22772","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Learning-Based BMD Estimation from Radiographs with Conformal Uncertainty Quantification","date":"2025-05-28","arxiv_id":"2505.22551","n_code_links":0,"syntology":null},{"paper":null,"title":"Multipath cycleGAN for harmonization of paired and unpaired low-dose lung computed tomography reconstruction kernels","date":"2025-05-28","arxiv_id":"2505.22568","n_code_links":0,"syntology":null},{"paper":null,"title":"Intelligent Incident Hypertension Prediction in Obstructive Sleep Apnea","date":"2025-05-27","arxiv_id":"2505.20615","n_code_links":0,"syntology":null},{"paper":null,"title":"Unpaired Image-to-Image Translation for Segmentation and Signal Unmixing","date":"2025-05-27","arxiv_id":"2505.20746","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Learning for Breast Cancer Detection: Comparative Analysis of ConvNeXT and EfficientNet","date":"2025-05-24","arxiv_id":"2505.18725","n_code_links":0,"syntology":null},{"paper":null,"title":"Anatomy-Guided Multitask Learning for MRI-Based Classification of Placenta Accreta Spectrum and its Subtypes","date":"2025-05-23","arxiv_id":"2505.17484","n_code_links":0,"syntology":null},{"paper":null,"title":"EVM-Fusion: An Explainable Vision Mamba Architecture with Neural Algorithmic Fusion","date":"2025-05-23","arxiv_id":"2505.17367","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing Federated Survival Analysis through Peer-Driven Client Reputation in Healthcare","date":"2025-05-22","arxiv_id":"2505.16190","n_code_links":0,"syntology":null},{"paper":null,"title":"Fusion of Foundation and Vision Transformer Model Features for Dermatoscopic Image Classification","date":"2025-05-22","arxiv_id":"2505.16338","n_code_links":0,"syntology":null},{"paper":"/paper/graph-supported-dynamic-algorithm","title":"Graph-Supported Dynamic Algorithm Configuration for Multi-Objective Combinatorial Optimization","date":"2025-05-22","arxiv_id":"2505.16471","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":null,"title":"Higher-Order Asymptotics of Test-Time Adaptation for Batch Normalization Statistics","date":"2025-05-22","arxiv_id":"2505.16257","n_code_links":0,"syntology":null},{"paper":null,"title":"InspectionV3: Enhancing Tobacco Quality Assessment with Deep Convolutional Neural Networks for Automated Workshop Management","date":"2025-05-22","arxiv_id":"2505.16485","n_code_links":0,"syntology":null},{"paper":null,"title":"SuperPure: Efficient Purification of Localized and Distributed Adversarial Patches via Super-Resolution GAN Models","date":"2025-05-22","arxiv_id":"2505.16318","n_code_links":0,"syntology":null},{"paper":null,"title":"An Approach Towards Identifying Bangladeshi Leaf Diseases through Transfer Learning and XAI","date":"2025-05-21","arxiv_id":"2505.16033","n_code_links":0,"syntology":null},{"paper":null,"title":"An Exploratory Approach Towards Investigating and Explaining Vision Transformer and Transfer Learning for Brain Disease Detection","date":"2025-05-21","arxiv_id":"2505.16039","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":702},{"task":"/task/object-detection","name":"Object Detection","papers":608},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":598},{"task":"/task/object-detection-1","name":"object-detection","papers":548},{"task":"/task/image-classification","name":"image-classification","papers":536},{"task":"/task/segmentation","name":"Segmentation","papers":408},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":344},{"task":"/task/classification","name":"General Classification","papers":320},{"task":"/task/classification-1","name":"Classification","papers":300},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":268},{"task":"/task/object","name":"Object","papers":257},{"task":"/task/deep-learning","name":"Deep Learning","papers":220},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":209},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":207},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":197},{"task":"/task/image-generation","name":"Image Generation","papers":190},{"task":"/task/representation-learning","name":"Representation Learning","papers":189},{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":187},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":182},{"task":"/task/quantization","name":"Quantization","papers":180}],"tasks_shown":20,"n_tasks":1316,"usage_by_year":[{"year":"2015","papers":14},{"year":"2016","papers":72},{"year":"2017","papers":194},{"year":"2018","papers":439},{"year":"2019","papers":729},{"year":"2020","papers":1039},{"year":"2021","papers":1107},{"year":"2022","papers":942},{"year":"2023","papers":879},{"year":"2024","papers":655},{"year":"2025","papers":217}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/batch-normalization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}