{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-images-from-caption-and-vice-versa","title":"Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search","arxiv_id":"2102.01645","date":"2021-02-02","proceeding":null,"authors":["Federico A. Galatolo","Mario G. C. A. Cimino","Gigliola Vaglini"],"abstract":"In this research work we present CLIP-GLaSS, a novel zero-shot framework to generate an image (or a caption) corresponding to a given caption (or image). CLIP-GLaSS is based on the CLIP neural network, which, given an image and a descriptive caption, provides similar embeddings. Differently, CLIP-GLaSS takes a caption (or an image) as an input, and generates the image (or the caption) whose CLIP embedding is the most similar to the input one. This optimal image (or caption) is produced via a generative network, after an exploration by a genetic algorithm. Promising results are shown, based on the experimentation of the image Generators BigGAN and StyleGAN2, and of the text Generator GPT2","url_abs":"https://arxiv.org/abs/2102.01645v4","url_pdf":"https://arxiv.org/pdf/2102.01645v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-images-from-caption-and-vice-versa","repo_url":"https://github.com/galatolofederico/clip-glass","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"generating-images-from-caption-and-vice-versa","repo_url":"https://github.com/armaank/archlectures","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"generating-images-from-caption-and-vice-versa","repo_url":"https://github.com/yk/clip_music_video","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"biggan","method_name":"BigGAN"},{"method_slug":"conditional-batch-normalization","method_name":"Conditional Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"early-stopping","method_name":"Early Stopping"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"gan-hinge-loss","method_name":"GAN Hinge Loss"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"non-local-block","method_name":"Non-Local Block"},{"method_slug":"non-local-operation","method_name":"Non-Local Operation"},{"method_slug":"off-diagonal-orthogonal-regularization","method_name":"Off-Diagonal Orthogonal Regularization"},{"method_slug":"path-length-regularization","method_name":"Path Length Regularization"},{"method_slug":"projection-discriminator","method_name":"Projection Discriminator"},{"method_slug":"r1-regularization","method_name":"R1 Regularization"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sagan","method_name":"SAGAN"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spectral-normalization","method_name":"Spectral Normalization"},{"method_slug":"stylegan2","method_name":"StyleGAN2"},{"method_slug":"ttur","method_name":"TTUR"},{"method_slug":"truncation-trick","method_name":"Truncation Trick"},{"method_slug":"weight-demodulation","method_name":"Weight Demodulation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.01645","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}