{"url":"/method/conditional-batch-normalization","slug":"conditional-batch-normalization","name":"Conditional Batch Normalization","full_name":"Conditional Batch Normalization","full_name_withheld":false,"description_markdown":"**Conditional Batch Normalization (CBN)** is a class-conditional variant of [batch normalization](https://paperswithcode.com/method/batch-normalization). The key idea is to predict the $\\gamma$ and $\\beta$ of the batch normalization from an embedding - e.g. a language embedding in VQA. CBN enables the linguistic embedding to manipulate entire feature maps by scaling them up or down, negating them, or shutting them off. CBN has also been used in [GANs](https://paperswithcode.com/methods/category/generative-adversarial-networks) to allow class information to affect the batch normalization parameters.\r\n\r\nConsider a single convolutional layer with batch normalization module $\\text{BN}\\left(F\\_{i,c,h,w}|\\gamma\\_{c}, \\beta\\_{c}\\right)$ for which pretrained scalars $\\gamma\\_{c}$ and $\\beta\\_{c}$ are available. We would like to directly predict these affine scaling parameters from, e.g., a language embedding $\\mathbf{e\\_{q}}$. When starting the training procedure, these parameters must be close to the pretrained values to recover the original [ResNet](https://paperswithcode.com/method/resnet) model as a poor initialization could significantly deteriorate performance. Unfortunately, it is difficult to initialize a network to output the pretrained $\\gamma$ and $\\beta$. For these reasons, the authors propose to predict a change $\\delta\\beta\\_{c}$ and $\\delta\\gamma\\_{c}$ on the frozen original scalars, for which it is straightforward to initialize a neural network to produce an output with zero-mean and small variance.\r\n\r\nThe authors use a one-hidden-layer MLP to predict these deltas from a question embedding $\\mathbf{e\\_{q}}$ for all feature maps within the layer:\r\n\r\n$$\\Delta\\beta = \\text{MLP}\\left(\\mathbf{e\\_{q}}\\right)$$\r\n\r\n$$\\Delta\\gamma = \\text{MLP}\\left(\\mathbf{e\\_{q}}\\right)$$\r\n\r\nSo, given a feature map with $C$ channels, these MLPs output a vector of size $C$. We then add these predictions to the $\\beta$ and $\\gamma$ parameters:\r\n\r\n$$ \\hat{\\beta}\\_{c} = \\beta\\_{c} + \\Delta\\beta\\_{c} $$\r\n\r\n$$ \\hat{\\gamma}\\_{c} = \\gamma\\_{c} + \\Delta\\gamma\\_{c} $$\r\n\r\nFinally, these updated $\\hat{β}$ and $\\hat{\\gamma}$ are used as parameters for the batch normalization: $\\text{BN}\\left(F\\_{i,c,h,w}|\\hat{\\gamma\\_{c}}, \\hat{\\beta\\_{c}}\\right)$. The authors freeze all ResNet parameters, including $\\gamma$ and $\\beta$, during training. A ResNet consists of\r\nfour stages of computation, each subdivided in several residual blocks. In each block, the authors apply CBN to the three convolutional layers.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Modulating early visual processing by language","paper":"/paper/modulating-early-visual-processing-by","first_author":"Harm de Vries","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/modulating-early-visual-processing-by"},"source":{"url":"http://arxiv.org/abs/1707.00683v3","title":"Modulating early visual processing by language","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/ap229997/Conditional-Batch-Norm/blob/6e237ed5794246e1bbbe95bbda9acf81d0cdeace/model/cbn.py#L9","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Normalization","url":"/methods/category/normalization","pwc_aliases":[]}],"n_papers_tagged":145,"archive_num_papers":145,"papers_newest_first":[{"paper":null,"title":"ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks","date":"2024-11-06","arxiv_id":"2411.03999","n_code_links":0,"syntology":null},{"paper":"/paper/unsupervised-panoptic-interpretation-of","title":"Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantization","date":"2024-10-27","arxiv_id":"2410.20573","n_code_links":1,"syntology":null},{"paper":"/paper/ratlip-generative-adversarial-clip-text-to","title":"RATLIP: Generative Adversarial CLIP Text-to-Image Synthesis Based on Recurrent Affine Transformations","date":"2024-05-13","arxiv_id":"2405.08114","n_code_links":1,"syntology":null},{"paper":"/paper/data-driven-crop-growth-simulation-on-time","title":"Data-driven Crop Growth Simulation on Time-varying Generated Images using Multi-conditional Generative Adversarial Networks","date":"2023-12-06","arxiv_id":"2312.03443","n_code_links":1,"syntology":null},{"paper":null,"title":"On quantifying and improving realism of images generated with diffusion","date":"2023-09-26","arxiv_id":"2309.14756","n_code_links":0,"syntology":null},{"paper":null,"title":"Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flows","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/a-strategic-framework-for-optimal-decisions","title":"A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theory","date":"2023-07-27","arxiv_id":"2307.14732","n_code_links":2,"syntology":null},{"paper":"/paper/pyrus-base-an-open-source-python-framework","title":"Pyrus Base: An Open Source Python Framework for the RoboCup 2D Soccer Simulation","date":"2023-07-22","arxiv_id":"2307.16875","n_code_links":1,"syntology":null},{"paper":"/paper/diffusion-models-beat-gans-on-image","title":"Diffusion Models Beat GANs on Image Classification","date":"2023-07-17","arxiv_id":"2307.08702","n_code_links":1,"syntology":null},{"paper":null,"title":"Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs","date":"2023-06-28","arxiv_id":"2306.15903","n_code_links":0,"syntology":null},{"paper":null,"title":"Rosetta Neurons: Mining the Common Units in a Model Zoo","date":"2023-06-15","arxiv_id":"2306.09346","n_code_links":0,"syntology":null},{"paper":null,"title":"Toward more accurate and generalizable brain deformation estimators for traumatic brain injury detection with unsupervised domain adaptation","date":"2023-06-08","arxiv_id":"2306.05255","n_code_links":0,"syntology":null},{"paper":null,"title":"FOOCTTS: Generating Arabic Speech with Acoustic Environment for Football Commentator","date":"2023-06-07","arxiv_id":"2306.07936","n_code_links":0,"syntology":null},{"paper":null,"title":"Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning","date":"2023-05-29","arxiv_id":"2305.17886","n_code_links":0,"syntology":null},{"paper":"/paper/is-centralized-training-with-decentralized","title":"Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?","date":"2023-05-27","arxiv_id":"2305.17352","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":1}},{"paper":null,"title":"Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations","date":"2023-05-22","arxiv_id":"2305.13030","n_code_links":0,"syntology":null},{"paper":"/paper/an-empirical-study-on-google-research","title":"An Empirical Study on Google Research Football Multi-agent Scenarios","date":"2023-05-16","arxiv_id":"2305.09458","n_code_links":1,"syntology":null},{"paper":"/paper/the-muse-2023-multimodal-sentiment-analysis","title":"The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions, Cross-Cultural Humour, and Personalisation","date":"2023-05-05","arxiv_id":"2305.03369","n_code_links":1,"syntology":null},{"paper":"/paper/sportsmot-a-large-multi-object-tracking","title":"SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes","date":"2023-04-11","arxiv_id":"2304.05170","n_code_links":1,"syntology":null},{"paper":"/paper/vars-video-assistant-referee-system-for","title":"VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views","date":"2023-04-10","arxiv_id":"2304.04617","n_code_links":1,"syntology":null},{"paper":null,"title":"Towards Active Learning for Action Spotting in Association Football Videos","date":"2023-04-09","arxiv_id":"2304.04220","n_code_links":0,"syntology":null},{"paper":null,"title":"Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football","date":"2023-03-25","arxiv_id":"2303.15471","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Generative Multi-Agent Imitation Model as a Computational Benchmark for Evaluating Human Performance in Complex Interactive Tasks: A Case Study in Football","date":"2023-03-23","arxiv_id":"2303.13323","n_code_links":0,"syntology":null},{"paper":"/paper/high-fidelity-image-synthesis-with-deep-vaes","title":"High Fidelity Image Synthesis With Deep VAEs In Latent Space","date":"2023-03-23","arxiv_id":"2303.13714","n_code_links":2,"syntology":{"ran":2,"of":2,"unverified":0,"pointer_only":2}},{"paper":"/paper/breaking-common-sense-whoops-a-vision-and","title":"Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images","date":"2023-03-13","arxiv_id":"2303.07274","n_code_links":0,"syntology":null},{"paper":"/paper/nfl-career-success-as-predicted-by-nfl","title":"NFL Career Success as Predicted by NFL Scouting Combine","date":"2023-03-10","arxiv_id":"2303.05774","n_code_links":1,"syntology":null},{"paper":"/paper/ultra-high-resolution-detector-simulation","title":"Ultra-High-Resolution Detector Simulation with Intra-Event Aware GAN and Self-Supervised Relational Reasoning","date":"2023-03-07","arxiv_id":"2303.08046","n_code_links":1,"syntology":null},{"paper":null,"title":"Identification of pattern mining algorithm for rugby league players positional groups separation based on movement patterns","date":"2023-02-25","arxiv_id":"2302.14058","n_code_links":0,"syntology":null},{"paper":"/paper/transformer-based-neural-marked-spatio","title":"Transformer-Based Neural Marked Spatio Temporal Point Process Model for Football Match Events Analysis","date":"2023-02-18","arxiv_id":"2302.09276","n_code_links":1,"syntology":null},{"paper":"/paper/tizero-mastering-multi-agent-football-with","title":"TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play","date":"2023-02-15","arxiv_id":"2302.07515","n_code_links":1,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/image-generation","name":"Image Generation","papers":42},{"task":"/task/conditional-image-generation","name":"Conditional Image Generation","papers":16},{"task":null,"name":"Generative Adversarial Network","papers":12},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":9},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":7},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":7},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":6},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":6},{"task":"/task/super-resolution","name":"Super-Resolution","papers":6},{"task":"/task/high","name":"Vocal Bursts Intensity Prediction","papers":6},{"task":"/task/decision-making","name":"Decision Making","papers":5},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":5},{"task":"/task/unconditional-image-generation","name":"Unconditional Image Generation","papers":5},{"task":"/task/attribute","name":"Attribute","papers":4},{"task":"/task/clustering","name":"Clustering","papers":4},{"task":"/task/denoising","name":"Denoising","papers":4},{"task":"/task/diversity","name":"Diversity","papers":4},{"task":"/task/object","name":"Object","papers":4},{"task":"/task/benchmarking","name":"Benchmarking","papers":3},{"task":"/task/decoder","name":"Decoder","papers":3}],"tasks_shown":20,"n_tasks":145,"usage_by_year":[{"year":"2017","papers":2},{"year":"2018","papers":4},{"year":"2019","papers":11},{"year":"2020","papers":31},{"year":"2021","papers":21},{"year":"2022","papers":35},{"year":"2023","papers":38},{"year":"2024","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/conditional-batch-normalization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}