{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/consistency-regularization-for-generative-1","title":"Consistency Regularization for Generative Adversarial Networks","arxiv_id":"1910.12027","date":"2019-10-26","proceeding":"ICLR 2020 1","authors":["Han Zhang","Zizhao Zhang","Augustus Odena","Honglak Lee"],"abstract":"Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization. In this work, we propose a simple, effective training stabilizer based on the notion of consistency regularization---a popular technique in the semi-supervised learning literature. In particular, we augment data passing into the GAN discriminator and penalize the sensitivity of the discriminator to these augmentations. We conduct a series of experiments to demonstrate that consistency regularization works effectively with spectral normalization and various GAN architectures, loss functions and optimizer settings. Our method achieves the best FID scores for unconditional image generation compared to other regularization methods on CIFAR-10 and CelebA. Moreover, Our consistency regularized GAN (CR-GAN) improves state-of-the-art FID scores for conditional generation from 14.73 to 11.48 on CIFAR-10 and from 8.73 to 6.66 on ImageNet-2012.","url_abs":"https://arxiv.org/abs/1910.12027v2","url_pdf":"https://arxiv.org/pdf/1910.12027v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"conditional-image-generation","task_name":"Conditional Image Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"unconditional-image-generation","task_name":"Unconditional Image Generation"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"spectral-normalization","method_name":"Spectral Normalization"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/conditional-image-generation-on-artbench-10","task":"Conditional Image Generation","dataset":"ArtBench-10 (32x32)","model":"BigGAN + CR","rank_in_archive_order":5,"of":6,"metrics":{"FID":"4.647"},"uses_additional_data":false},{"leaderboard":"/sota/conditional-image-generation-on-cifar-10","task":"Conditional Image Generation","dataset":"CIFAR-10","model":"CR-BigGAN","rank_in_archive_order":13,"of":25,"metrics":{"FID":"11.67"},"uses_additional_data":false},{"leaderboard":"/sota/conditional-image-generation-on-imagenet","task":"Conditional Image Generation","dataset":"ImageNet 128x128","model":"CR-BigGAN","rank_in_archive_order":9,"of":22,"metrics":{"FID":"6.66"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-celeba-hq-128x128","task":"Image Generation","dataset":"CelebA-HQ 128x128","model":"CR-GAN","rank_in_archive_order":5,"of":7,"metrics":{"FID":"16.97"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-imagenet-128x128","task":"Image Generation","dataset":"ImageNet 128x128","model":"CR-BigGAN","rank_in_archive_order":15,"of":23,"metrics":{"FID":"6.66"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1910.12027","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}