{"url":"/method/spectral-normalization","slug":"spectral-normalization","name":"Spectral Normalization","full_name":"Spectral Normalization","full_name_withheld":false,"description_markdown":"**Spectral Normalization** is a normalization technique used for generative adversarial networks, used to stabilize training of the discriminator. Spectral normalization has the convenient property that the Lipschitz constant is the only hyper-parameter to be tuned.\r\n\r\nIt controls the Lipschitz constant of the discriminator $f$ by constraining the spectral norm of each layer $g : \\textbf{h}\\_{in} \\rightarrow \\textbf{h}_{out}$. The Lipschitz norm $\\Vert{g}\\Vert\\_{\\text{Lip}}$ is equal to $\\sup\\_{\\textbf{h}}\\sigma\\left(\\nabla{g}\\left(\\textbf{h}\\right)\\right)$, where $\\sigma\\left(a\\right)$ is the spectral norm of the matrix $A$ ($L\\_{2}$ matrix norm of $A$):\r\n\r\n$$ \\sigma\\left(a\\right) = \\max\\_{\\textbf{h}:\\textbf{h}\\neq{0}}\\frac{\\Vert{A\\textbf{h}}\\Vert\\_{2}}{\\Vert\\textbf{h}\\Vert\\_{2}} = \\max\\_{\\Vert\\textbf{h}\\Vert\\_{2}\\leq{1}}{\\Vert{A\\textbf{h}}\\Vert\\_{2}} $$\r\n\r\nwhich is equivalent to the largest singular value of $A$. Therefore for a [linear layer](https://paperswithcode.com/method/linear-layer) $g\\left(\\textbf{h}\\right) = W\\textbf{h}$ the norm is given by $\\Vert{g}\\Vert\\_{\\text{Lip}} = \\sup\\_{\\textbf{h}}\\sigma\\left(\\nabla{g}\\left(\\textbf{h}\\right)\\right) = \\sup\\_{\\textbf{h}}\\sigma\\left(W\\right) = \\sigma\\left(W\\right) $. Spectral normalization normalizes the spectral norm of the weight matrix $W$ so it satisfies the Lipschitz constraint $\\sigma\\left(W\\right) = 1$:\r\n\r\n$$ \\bar{W}\\_{\\text{SN}}\\left(W\\right) = W / \\sigma\\left(W\\right) $$","description_state":"present","introduced_year":null,"introduced_by":{"title":"Spectral Normalization for Generative Adversarial Networks","paper":"/paper/spectral-normalization-for-generative","first_author":"Takeru Miyato","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/spectral-normalization-for-generative"},"source":{"url":"http://arxiv.org/abs/1802.05957v1","title":"Spectral Normalization for Generative Adversarial Networks","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/christiancosgrove/pytorch-spectral-normalization-gan/blob/12dcf945a6359301d63d1e0da3708cd0f0590b19/spectral_normalization.py#L14","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Normalization","url":"/methods/category/normalization","pwc_aliases":[]}],"n_papers_tagged":220,"archive_num_papers":220,"papers_newest_first":[{"paper":null,"title":"Unpaired Image-to-Image Translation for Segmentation and Signal Unmixing","date":"2025-05-27","arxiv_id":"2505.20746","n_code_links":0,"syntology":null},{"paper":null,"title":"Spectral Normalization for Lipschitz-Constrained Policies on Learning Humanoid Locomotion","date":"2025-04-11","arxiv_id":"2504.08246","n_code_links":0,"syntology":null},{"paper":null,"title":"Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks","date":"2025-01-08","arxiv_id":"2501.04816","n_code_links":0,"syntology":null},{"paper":null,"title":"ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks","date":"2024-11-06","arxiv_id":"2411.03999","n_code_links":0,"syntology":null},{"paper":"/paper/unsupervised-panoptic-interpretation-of","title":"Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantization","date":"2024-10-27","arxiv_id":"2410.20573","n_code_links":1,"syntology":null},{"paper":null,"title":"Normalizing self-supervised learning for provably reliable Change Point Detection","date":"2024-10-17","arxiv_id":"2410.13637","n_code_links":0,"syntology":null},{"paper":null,"title":"XMOL: Explainable Multi-property Optimization of Molecules","date":"2024-09-12","arxiv_id":"2409.07786","n_code_links":0,"syntology":null},{"paper":"/paper/particle-physics-dl-simulation-with-control","title":"Particle physics DL-simulation with control over generated data properties","date":"2024-05-22","arxiv_id":"2405.14049","n_code_links":1,"syntology":null},{"paper":"/paper/spatial-aware-attention-generative","title":"Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image","date":"2024-05-21","arxiv_id":"2405.12872","n_code_links":1,"syntology":null},{"paper":"/paper/on-robust-reinforcement-learning-with","title":"On Robust Reinforcement Learning with Lipschitz-Bounded Policy Networks","date":"2024-05-19","arxiv_id":"2405.11432","n_code_links":2,"syntology":null},{"paper":"/paper/multi-label-out-of-distribution-detection","title":"Multi-Label Out-of-Distribution Detection with Spectral Normalized Joint Energy","date":"2024-05-08","arxiv_id":"2405.04759","n_code_links":1,"syntology":null},{"paper":"/paper/adaptive-catalyst-discovery-using","title":"Adaptive Catalyst Discovery Using Multicriteria Bayesian Optimization with Representation Learning","date":"2024-04-18","arxiv_id":"2404.12445","n_code_links":1,"syntology":null},{"paper":"/paper/policy-bifurcation-in-safe-reinforcement","title":"Policy Bifurcation in Safe Reinforcement Learning","date":"2024-03-19","arxiv_id":"2403.12847","n_code_links":1,"syntology":null},{"paper":null,"title":"BFRFormer: Transformer-based generator for Real-World Blind Face Restoration","date":"2024-02-29","arxiv_id":"2402.18811","n_code_links":0,"syntology":null},{"paper":null,"title":"Control-Theoretic Techniques for Online Adaptation of Deep Neural Networks in Dynamical Systems","date":"2024-02-01","arxiv_id":"2402.00761","n_code_links":0,"syntology":null},{"paper":null,"title":"Convergences for Minimax Optimization Problems over Infinite-Dimensional Spaces Towards Stability in Adversarial Training","date":"2023-12-02","arxiv_id":"2312.00991","n_code_links":0,"syntology":null},{"paper":null,"title":"On quantifying and improving realism of images generated with diffusion","date":"2023-09-26","arxiv_id":"2309.14756","n_code_links":0,"syntology":null},{"paper":"/paper/model-based-reparameterization-policy","title":"Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithms","date":"2023-09-21","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flows","date":"2023-09-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/a-strategic-framework-for-optimal-decisions","title":"A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theory","date":"2023-07-27","arxiv_id":"2307.14732","n_code_links":2,"syntology":null},{"paper":"/paper/pyrus-base-an-open-source-python-framework","title":"Pyrus Base: An Open Source Python Framework for the RoboCup 2D Soccer Simulation","date":"2023-07-22","arxiv_id":"2307.16875","n_code_links":1,"syntology":null},{"paper":"/paper/diffusion-models-beat-gans-on-image","title":"Diffusion Models Beat GANs on Image Classification","date":"2023-07-17","arxiv_id":"2307.08702","n_code_links":1,"syntology":null},{"paper":null,"title":"A Bayesian approach to quantifying uncertainties and improving generalizability in traffic prediction models","date":"2023-07-12","arxiv_id":"2307.05946","n_code_links":0,"syntology":null},{"paper":null,"title":"Diversity is Strength: Mastering Football Full Game with Interactive Reinforcement Learning of Multiple AIs","date":"2023-06-28","arxiv_id":"2306.15903","n_code_links":0,"syntology":null},{"paper":null,"title":"Rosetta Neurons: Mining the Common Units in a Model Zoo","date":"2023-06-15","arxiv_id":"2306.09346","n_code_links":0,"syntology":null},{"paper":null,"title":"Toward more accurate and generalizable brain deformation estimators for traumatic brain injury detection with unsupervised domain adaptation","date":"2023-06-08","arxiv_id":"2306.05255","n_code_links":0,"syntology":null},{"paper":null,"title":"FOOCTTS: Generating Arabic Speech with Acoustic Environment for Football Commentator","date":"2023-06-07","arxiv_id":"2306.07936","n_code_links":0,"syntology":null},{"paper":"/paper/relu-to-the-rescue-improve-your-on-policy","title":"ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages","date":"2023-06-02","arxiv_id":"2306.01460","n_code_links":1,"syntology":null},{"paper":null,"title":"Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning","date":"2023-05-29","arxiv_id":"2305.17886","n_code_links":0,"syntology":null},{"paper":"/paper/is-centralized-training-with-decentralized","title":"Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?","date":"2023-05-27","arxiv_id":"2305.17352","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":1}}],"papers_shown":30,"tasks":[{"task":"/task/image-generation","name":"Image Generation","papers":57},{"task":"/task/conditional-image-generation","name":"Conditional Image Generation","papers":20},{"task":null,"name":"Generative Adversarial Network","papers":19},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":17},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":13},{"task":"/task/translation","name":"Translation","papers":13},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":12},{"task":"/task/image-to-image-translation","name":"Image-to-Image Translation","papers":11},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":10},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":9},{"task":"/task/super-resolution","name":"Super-Resolution","papers":9},{"task":"/task/image-classification","name":"Image Classification","papers":7},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":7},{"task":"/task/unconditional-image-generation","name":"Unconditional Image Generation","papers":7},{"task":"/task/high","name":"Vocal Bursts Intensity Prediction","papers":7},{"task":"/task/image-classification","name":"image-classification","papers":7},{"task":"/task/attribute","name":"Attribute","papers":6},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":6},{"task":"/task/decision-making","name":"Decision Making","papers":5},{"task":"/task/denoising","name":"Denoising","papers":5}],"tasks_shown":20,"n_tasks":201,"usage_by_year":[{"year":"2018","papers":14},{"year":"2019","papers":22},{"year":"2020","papers":48},{"year":"2021","papers":37},{"year":"2022","papers":41},{"year":"2023","papers":43},{"year":"2024","papers":12},{"year":"2025","papers":3}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/spectral-normalization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}