{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusion-models-beat-gans-on-image","title":"Diffusion Models Beat GANs on Image Classification","arxiv_id":"2307.08702","date":"2023-07-17","proceeding":null,"authors":["Soumik Mukhopadhyay","Matthew Gwilliam","Vatsal Agarwal","Namitha Padmanabhan","Archana Swaminathan","Srinidhi Hegde","Tianyi Zhou","Abhinav Shrivastava"],"abstract":"While many unsupervised learning models focus on one family of tasks, either generative or discriminative, we explore the possibility of a unified representation learner: a model which uses a single pre-training stage to address both families of tasks simultaneously. We identify diffusion models as a prime candidate. Diffusion models have risen to prominence as a state-of-the-art method for image generation, denoising, inpainting, super-resolution, manipulation, etc. Such models involve training a U-Net to iteratively predict and remove noise, and the resulting model can synthesize high fidelity, diverse, novel images. The U-Net architecture, as a convolution-based architecture, generates a diverse set of feature representations in the form of intermediate feature maps. We present our findings that these embeddings are useful beyond the noise prediction task, as they contain discriminative information and can also be leveraged for classification. We explore optimal methods for extracting and using these embeddings for classification tasks, demonstrating promising results on the ImageNet classification task. We find that with careful feature selection and pooling, diffusion models outperform comparable generative-discriminative methods such as BigBiGAN for classification tasks. We investigate diffusion models in the transfer learning regime, examining their performance on several fine-grained visual classification datasets. We compare these embeddings to those generated by competing architectures and pre-trainings for classification tasks.","url_abs":"https://arxiv.org/abs/2307.08702v1","url_pdf":"https://arxiv.org/pdf/2307.08702v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusion-models-beat-gans-on-image","repo_url":"https://github.com/soumik-kanad/diffssl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"fine-grained-image-classification","task_name":"Fine-Grained Image Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"super-resolution","task_name":"Super-Resolution"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"feature-selection","task_name":"feature selection"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bigbigan","method_name":"BigBiGAN"},{"method_slug":"biggan","method_name":"BigGAN"},{"method_slug":"crelu","method_name":"CReLU"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"conditional-batch-normalization","method_name":"Conditional Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"early-stopping","method_name":"Early Stopping"},{"method_slug":"feature-selection","method_name":"Feature Selection"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"gan-hinge-loss","method_name":"GAN Hinge Loss"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"non-local-block","method_name":"Non-Local Block"},{"method_slug":"non-local-operation","method_name":"Non-Local Operation"},{"method_slug":"off-diagonal-orthogonal-regularization","method_name":"Off-Diagonal Orthogonal Regularization"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"projection-discriminator","method_name":"Projection Discriminator"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"revnet","method_name":"RevNet"},{"method_slug":"reversible-residual-block","method_name":"Reversible Residual Block"},{"method_slug":"sagan","method_name":"SAGAN"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spectral-normalization","method_name":"Spectral Normalization"},{"method_slug":"ttur","method_name":"TTUR"},{"method_slug":"truncation-trick","method_name":"Truncation Trick"},{"method_slug":"u-net","method_name":"U-Net"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.08702","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}