{"url":"/method/gradient-normalization","slug":"gradient-normalization","name":"Gradient Normalization","full_name":"Gradient Normalization","full_name_withheld":false,"description_markdown":"**Gradient Normalization** is a normalization method for [Generative Adversarial Networks](https://paperswithcode.com/methods/category/generative-adversarial-networks) to tackle the training instability of generative adversarial networks caused by the sharp gradient space. Unlike existing work such as [gradient penalty](https://paperswithcode.com/method/wgan-gp-loss) and [spectral normalization](https://paperswithcode.com/method/spectral-normalization), the proposed GN only imposes a hard 1-Lipschitz constraint on the discriminator function, which increases the capacity of the network.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/2109.02235v2","title":"Gradient Normalization for Generative Adversarial Networks","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Normalization","url":"/methods/category/normalization","pwc_aliases":[]}],"n_papers_tagged":15,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"AlphaGrad: Non-Linear Gradient Normalization Optimizer","date":"2025-04-22","arxiv_id":"2504.16020","n_code_links":0,"syntology":null},{"paper":"/paper/stable-spam-how-to-train-in-4-bit-more-stably","title":"Stable-SPAM: How to Train in 4-Bit More Stably than 16-Bit Adam","date":"2025-02-24","arxiv_id":"2502.17055","n_code_links":1,"syntology":null},{"paper":null,"title":"Network scaling and scale-driven loss balancing for intelligent poroelastography","date":"2024-10-27","arxiv_id":"2411.08886","n_code_links":0,"syntology":null},{"paper":null,"title":"Gradient Normalization Provably Benefits Nonconvex SGD under Heavy-Tailed Noise","date":"2024-10-21","arxiv_id":"2410.16561","n_code_links":0,"syntology":null},{"paper":null,"title":"Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization","date":"2024-10-17","arxiv_id":"2410.14054","n_code_links":0,"syntology":null},{"paper":null,"title":"Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation","date":"2024-10-11","arxiv_id":"2410.09125","n_code_links":0,"syntology":null},{"paper":null,"title":"Adaptive Gradient Regularization: A Faster and Generalizable Optimization Technique for Deep Neural Networks","date":"2024-07-24","arxiv_id":"2407.16944","n_code_links":0,"syntology":null},{"paper":"/paper/ofvl-ms-once-for-visual-localization-across","title":"OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes","date":"2023-08-23","arxiv_id":"2308.11928","n_code_links":1,"syntology":{"ran":3,"of":3,"unverified":0,"pointer_only":3}},{"paper":"/paper/continual-domain-adaptation-on-aerial-images","title":"Continual Domain Adaptation on Aerial Images under Gradually Degrading Weather","date":"2023-08-02","arxiv_id":"2308.00924","n_code_links":1,"syntology":null},{"paper":"/paper/penalty-gradient-normalization-for-generative","title":"Penalty Gradient Normalization for Generative Adversarial Networks","date":"2023-06-23","arxiv_id":"2306.13576","n_code_links":1,"syntology":null},{"paper":null,"title":"Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima","date":"2023-06-16","arxiv_id":"2306.09850","n_code_links":0,"syntology":null},{"paper":"/paper/a-deep-learning-approach-using-masked-image","title":"A Deep Learning Approach Using Masked Image Modeling for Reconstruction of Undersampled K-spaces","date":"2022-08-24","arxiv_id":"2208.11472","n_code_links":1,"syntology":null},{"paper":null,"title":"GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks","date":"2021-11-04","arxiv_id":"2111.03162","n_code_links":0,"syntology":null},{"paper":"/paper/update-in-unit-gradient","title":"Perturbated Gradients Updating within Unit Space for Deep Learning","date":"2021-10-01","arxiv_id":"2110.00199","n_code_links":1,"syntology":null},{"paper":"/paper/gradient-normalization-for-generative","title":"Gradient Normalization for Generative Adversarial Networks","date":"2021-09-06","arxiv_id":"2109.02235","n_code_links":1,"syntology":null}],"papers_shown":15,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/image-generation","name":"Image Generation","papers":2},{"task":"/task/image-classification","name":"image-classification","papers":2},{"task":"/task/all","name":"All","papers":1},{"task":"/task/binary-classification","name":"Binary Classification","papers":1},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/domain-adaptation","name":"Domain Adaptation","papers":1},{"task":"/task/federated-learning","name":"Federated Learning","papers":1},{"task":"/task/mri-reconstruction","name":"MRI Reconstruction","papers":1},{"task":"/task/multi-task-learning","name":"Multi-Task Learning","papers":1},{"task":"/task/stochastic-optimization","name":"Stochastic Optimization","papers":1},{"task":"/task/test-time-adaptation","name":"Test-time Adaptation","papers":1},{"task":"/task/vertical-federated-learning","name":"Vertical Federated Learning","papers":1},{"task":"/task/visual-localization","name":"Visual Localization","papers":1}],"tasks_shown":14,"n_tasks":14,"usage_by_year":[{"year":"2021","papers":3},{"year":"2022","papers":1},{"year":"2023","papers":4},{"year":"2024","papers":5},{"year":"2025","papers":2}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/gradient-normalization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}