{"url":"/method/group-normalization","slug":"group-normalization","name":"Group Normalization","full_name":"Group Normalization","full_name_withheld":false,"description_markdown":"**Group Normalization** is a normalization layer that divides channels into groups and normalizes the features within each group. GN does not exploit the batch dimension, and its computation is independent of batch sizes. In the case where the group size is 1, it is equivalent to [Instance Normalization](https://paperswithcode.com/method/instance-normalization).\r\n\r\nAs motivation for the method, many classical features like SIFT and HOG had *group-wise* features and involved *group-wise normalization*. For example, a HOG vector is the outcome of several spatial cells where each cell is represented by a normalized orientation histogram.\r\n\r\nFormally, Group Normalization is defined as:\r\n\r\n$$ \\mu\\_{i} = \\frac{1}{m}\\sum\\_{k\\in\\mathcal{S}\\_{i}}x\\_{k} $$\r\n\r\n$$ \\sigma^{2}\\_{i} = \\frac{1}{m}\\sum\\_{k\\in\\mathcal{S}\\_{i}}\\left(x\\_{k}-\\mu\\_{i}\\right)^{2} $$\r\n\r\n$$ \\hat{x}\\_{i} = \\frac{x\\_{i} - \\mu\\_{i}}{\\sqrt{\\sigma^{2}\\_{i}+\\epsilon}} $$\r\n\r\nHere $x$ is the feature computed by a layer, and $i$ is an index. Formally, a Group Norm layer computes $\\mu$ and $\\sigma$ in a set $\\mathcal{S}\\_{i}$ defined as: $\\mathcal{S}\\_{i} = ${$k \\mid k\\_{N} = i\\_{N} ,\\lfloor\\frac{k\\_{C}}{C/G}\\rfloor = \\lfloor\\frac{I\\_{C}}{C/G}\\rfloor $}.\r\n\r\nHere $G$ is the number of groups, which is a pre-defined hyper-parameter ($G = 32$ by default). $C/G$ is the number of channels per group. $\\lfloor$ is the floor operation, and the final term means that the indexes $i$ and $k$ are in the same group of channels, assuming each group of channels are stored in a sequential order along the $C$ axis.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Group Normalization","paper":"/paper/group-normalization","first_author":"Yuxin Wu","n_authors":2,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/group-normalization"},"source":{"url":"http://arxiv.org/abs/1803.08494v3","title":"Group Normalization","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/pytorch/pytorch/blob/1c5c289b6218eb1026dcb5fd9738231401cfccea/torch/nn/modules/normalization.py#L177","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Normalization","url":"/methods/category/normalization","pwc_aliases":[]}],"n_papers_tagged":55,"archive_num_papers":55,"papers_newest_first":[{"paper":null,"title":"Dynamic Group Normalization: Spatio-Temporal Adaptation to Evolving Data Statistics","date":"2025-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Rethinking Normalization Strategies and Convolutional Kernels for Multimodal Image Fusion","date":"2024-11-15","arxiv_id":"2411.10036","n_code_links":0,"syntology":null},{"paper":"/paper/unsupervised-adaptive-normalization","title":"Unsupervised Adaptive Normalization","date":"2024-09-07","arxiv_id":"2409.04757","n_code_links":1,"syntology":null},{"paper":null,"title":"Exploring the Efficacy of Group-Normalization in Deep Learning Models for Alzheimer's Disease Classification","date":"2024-04-01","arxiv_id":"2404.00946","n_code_links":0,"syntology":null},{"paper":"/paper/training-free-pretrained-model-merging","title":"Training-Free Pretrained Model Merging","date":"2024-03-04","arxiv_id":"2403.01753","n_code_links":1,"syntology":{"ran":3,"of":4,"unverified":1,"pointer_only":4}},{"paper":null,"title":"ELA: Efficient Local Attention for Deep Convolutional Neural Networks","date":"2024-03-02","arxiv_id":"2403.01123","n_code_links":0,"syntology":null},{"paper":"/paper/on-sensitivity-and-robustness-of","title":"On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis","date":"2023-06-23","arxiv_id":"2306.13276","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":"/paper/adaptive-sparse-convolutional-networks-with","title":"Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Images","date":"2023-03-25","arxiv_id":"2303.14488","n_code_links":1,"syntology":{"ran":3,"of":6,"unverified":3,"pointer_only":0}},{"paper":null,"title":"Making Batch Normalization Great in Federated Deep Learning","date":"2023-03-12","arxiv_id":"2303.06530","n_code_links":0,"syntology":null},{"paper":"/paper/on-the-ideal-number-of-groups-for-isometric","title":"On the Ideal Number of Groups for Isometric Gradient Propagation","date":"2023-02-07","arxiv_id":"2302.03193","n_code_links":0,"syntology":null},{"paper":null,"title":"MGTUNet: An new UNet for colon nuclei instance segmentation and quantification","date":"2022-10-20","arxiv_id":"2210.10981","n_code_links":0,"syntology":null},{"paper":null,"title":"Kernel Normalized Convolutional Networks for Privacy-Preserving Machine Learning","date":"2022-09-30","arxiv_id":"2210.00053","n_code_links":0,"syntology":null},{"paper":"/paper/training-a-universal-instance-segmentation","title":"Training a universal instance segmentation network for live cell images of various cell types and imaging modalities","date":"2022-07-28","arxiv_id":"2207.14347","n_code_links":1,"syntology":null},{"paper":"/paper/understanding-and-improving-group","title":"Understanding and Improving Group Normalization","date":"2022-07-05","arxiv_id":"2207.01972","n_code_links":1,"syntology":null},{"paper":null,"title":"Domain Adaptation and Active Learning for Fine-Grained Recognition in the Field of Biodiversity","date":"2021-10-22","arxiv_id":"2110.11778","n_code_links":0,"syntology":null},{"paper":null,"title":"Exploring Heterogeneous Characteristics of Layers in ASR Models for More Efficient Training","date":"2021-10-08","arxiv_id":"2110.04267","n_code_links":0,"syntology":null},{"paper":"/paper/scalable-deeper-graph-neural-networks-for","title":"Scalable deeper graph neural networks for high-performance materials property prediction","date":"2021-09-25","arxiv_id":"2109.12283","n_code_links":1,"syntology":null},{"paper":null,"title":"NanoBatch Privacy: Enabling fast Differentially Private learning on the IPU","date":"2021-09-24","arxiv_id":"2109.12191","n_code_links":0,"syntology":null},{"paper":null,"title":"Benchmarking the Robustness of Instance Segmentation Models","date":"2021-09-02","arxiv_id":"2109.01123","n_code_links":0,"syntology":null},{"paper":"/paper/effect-of-large-scale-pre-training-on-full","title":"Effect of Pre-Training Scale on Intra- and Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images","date":"2021-05-31","arxiv_id":"2106.00116","n_code_links":1,"syntology":null},{"paper":"/paper/genesis-v2-inferring-unordered-object","title":"GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement","date":"2021-04-20","arxiv_id":"2104.09958","n_code_links":2,"syntology":null},{"paper":null,"title":"Adaptive Feature Fusion Network for Gaze Tracking in Mobile Tablets","date":"2021-03-20","arxiv_id":"2103.11119","n_code_links":0,"syntology":null},{"paper":"/paper/spectr-spectral-transformer-for-hyperspectral","title":"SpecTr: Spectral Transformer for Hyperspectral Pathology Image Segmentation","date":"2021-03-05","arxiv_id":"2103.03604","n_code_links":1,"syntology":null},{"paper":null,"title":"Batch Group Normalization","date":"2020-12-04","arxiv_id":"2012.02782","n_code_links":0,"syntology":null},{"paper":null,"title":"NLCA-Net v2 for Stereo Matching in ECCV'20 Robust Vision Challenge","date":"2020-11-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/high-resolution-weakly-supervised","title":"High resolution weakly supervised localization architectures for medical images","date":"2020-10-22","arxiv_id":"2010.11475","n_code_links":1,"syntology":null},{"paper":"/paper/byol-works-even-without-batch-statistics","title":"BYOL works even without batch statistics","date":"2020-10-20","arxiv_id":"2010.10241","n_code_links":3,"syntology":{"ran":5,"of":5,"unverified":0,"pointer_only":3}},{"paper":"/paper/group-whitening-balancing-learning-efficiency","title":"Group Whitening: Balancing Learning Efficiency and Representational Capacity","date":"2020-09-28","arxiv_id":"2009.13333","n_code_links":1,"syntology":{"ran":3,"of":5,"unverified":2,"pointer_only":0}},{"paper":"/paper/benchmarking-semi-supervised-federated","title":"Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models","date":"2020-08-26","arxiv_id":"2008.11364","n_code_links":1,"syntology":{"ran":4,"of":11,"unverified":7,"pointer_only":0}},{"paper":null,"title":"New Interpretations of Normalization Methods in Deep Learning","date":"2020-06-16","arxiv_id":"2006.09104","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":11},{"task":"/task/image-classification","name":"Image Classification","papers":10},{"task":"/task/object-detection","name":"Object Detection","papers":9},{"task":"/task/segmentation","name":"Segmentation","papers":9},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":8},{"task":"/task/object-detection-1","name":"object-detection","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":5},{"task":"/task/domain-adaptation","name":"Domain Adaptation","papers":4},{"task":"/task/federated-learning","name":"Federated Learning","papers":4},{"task":"/task/few-shot-learning","name":"Few-Shot Learning","papers":4},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":4},{"task":"/task/object","name":"Object","papers":4},{"task":"/task/representation-learning","name":"Representation Learning","papers":4},{"task":"/task/high","name":"Vocal Bursts Intensity Prediction","papers":4},{"task":"/task/deep-learning","name":"Deep Learning","papers":3},{"task":null,"name":"GPU","papers":3},{"task":"/task/image-generation","name":"Image Generation","papers":3},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":3},{"task":"/task/clustering","name":"Clustering","papers":2},{"task":"/task/fine-grained-image-classification","name":"Fine-Grained Image Classification","papers":2}],"tasks_shown":20,"n_tasks":72,"usage_by_year":[{"year":"2017","papers":1},{"year":"2018","papers":4},{"year":"2019","papers":14},{"year":"2020","papers":13},{"year":"2021","papers":9},{"year":"2022","papers":4},{"year":"2023","papers":4},{"year":"2024","papers":5},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/group-normalization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}