{"url":"/method/mixture-normalization","slug":"mixture-normalization","name":"Mixture Normalization","full_name":"Mixture Normalization","full_name_withheld":false,"description_markdown":"**Mixture Normalization** is normalization technique that relies on an approximation of the probability density function of the internal representations. Any continuous distribution can be approximated with arbitrary precision using a Gaussian Mixture Model (GMM). Hence, instead of computing one set of statistical measures from the entire population (of instances in the mini-batch) as [Batch Normalization](https://paperswithcode.com/method/batch-normalization) does, Mixture Normalization works on sub-populations which can be identified by disentangling modes of the distribution, estimated via GMM. \r\n\r\nWhile BN can only scale and/or shift the whole underlying probability density function, mixture normalization operates like a soft piecewise normalizing transform, capable of completely re-structuring the data distribution by independently scaling and/or shifting individual modes of distribution.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Training Faster by Separating Modes of Variation in Batch-normalized Models","paper":"/paper/training-faster-by-separating-modes-of","first_author":"Mahdi M. Kalayeh","n_authors":2,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/training-faster-by-separating-modes-of"},"source":{"url":"http://arxiv.org/abs/1806.02892v2","title":"Training Faster by Separating Modes of Variation in Batch-normalized Models","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Normalization","url":"/methods/category/normalization","pwc_aliases":[]}],"n_papers_tagged":4,"archive_num_papers":4,"papers_newest_first":[{"paper":null,"title":"Adaptative Context Normalization: A Boost for Deep Learning in Image Processing","date":"2024-09-07","arxiv_id":"2409.04759","n_code_links":0,"syntology":null},{"paper":"/paper/unsupervised-adaptive-normalization","title":"Unsupervised Adaptive Normalization","date":"2024-09-07","arxiv_id":"2409.04757","n_code_links":1,"syntology":null},{"paper":"/paper/cluster-based-normalization-layer-for-neural","title":"Enhancing Neural Network Representations with Prior Knowledge-Based Normalization","date":"2024-03-25","arxiv_id":"2403.16798","n_code_links":1,"syntology":null},{"paper":"/paper/training-faster-by-separating-modes-of","title":"Training Faster by Separating Modes of Variation in Batch-normalized Models","date":"2018-06-07","arxiv_id":"1806.02892","n_code_links":1,"syntology":null}],"papers_shown":4,"tasks":[{"task":"/task/clustering","name":"Clustering","papers":2},{"task":"/task/domain-adaptation","name":"Domain Adaptation","papers":2},{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/image-classification","name":"image-classification","papers":2},{"task":"/task/image-generation","name":"Image Generation","papers":1}],"tasks_shown":5,"n_tasks":5,"usage_by_year":[{"year":"2018","papers":1},{"year":"2024","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/mixture-normalization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}