{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kandinsky-an-improved-text-to-image-synthesis","title":"Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion","arxiv_id":"2310.03502","date":"2023-10-05","proceeding":null,"authors":["Anton Razzhigaev","Arseniy Shakhmatov","Anastasia Maltseva","Vladimir Arkhipkin","Igor Pavlov","Ilya Ryabov","Angelina Kuts","Alexander Panchenko","Andrey Kuznetsov","Denis Dimitrov"],"abstract":"Text-to-image generation is a significant domain in modern computer vision and has achieved substantial improvements through the evolution of generative architectures. Among these, there are diffusion-based models that have demonstrated essential quality enhancements. These models are generally split into two categories: pixel-level and latent-level approaches. We present Kandinsky1, a novel exploration of latent diffusion architecture, combining the principles of the image prior models with latent diffusion techniques. The image prior model is trained separately to map text embeddings to image embeddings of CLIP. Another distinct feature of the proposed model is the modified MoVQ implementation, which serves as the image autoencoder component. Overall, the designed model contains 3.3B parameters. We also deployed a user-friendly demo system that supports diverse generative modes such as text-to-image generation, image fusion, text and image fusion, image variations generation, and text-guided inpainting/outpainting. Additionally, we released the source code and checkpoints for the Kandinsky models. Experimental evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking our model as the top open-source performer in terms of measurable image generation quality.","url_abs":"https://arxiv.org/abs/2310.03502v1","url_pdf":"https://arxiv.org/pdf/2310.03502v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kandinsky-an-improved-text-to-image-synthesis","repo_url":"https://github.com/ai-forever/Kandinsky-2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"favor","method_name":"FAVOR+"},{"method_slug":"performer","method_name":"Performer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/text-to-image-generation-on-coco","task":"Text-to-Image Generation","dataset":"COCO (Common Objects in Context)","model":"Kandinsky","rank_in_archive_order":19,"of":69,"metrics":{"FID":"8.03"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.03502","atlas_url":"https://app.syntology.ai/?focus=2310.03502","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.03502"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/mseitzer/pytorch-fid","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ai-forever/Kandinsky-2","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ai-forever/MoVQGAN","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/deep-floyd/IF","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":4,"ran_honours":3,"ran_draft_wrong":3,"ran_fixture":2,"unverified":6},"by_repo_kind":{"official":{"samples":8,"ran":3,"repositories":1},"found_in_text":{"samples":10,"ran":9,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"f60bca05893fa428","entry":"AttentionPooling","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f60bca05893fa428"}},{"code_sha256_prefix":"317687cb15c56f85","entry":"QKVAttention","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"317687cb15c56f85"}},{"code_sha256_prefix":"9deab2a47adbf3b7","entry":"approx_standard_normal_cdf","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/losses.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/losses.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9deab2a47adbf3b7"}},{"code_sha256_prefix":"fe30479834a53868","entry":"attention","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe30479834a53868"}},{"code_sha256_prefix":"6cdc8772f5db6f48","entry":"betas_for_alpha_bar","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/gaussian_diffusion.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6cdc8772f5db6f48"}},{"code_sha256_prefix":"f702cecaa7981c73","entry":"discretized_gaussian_log_likelihood","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/losses.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f702cecaa7981c73"}},{"code_sha256_prefix":"f51cf2cbd3e598d8","entry":"gelu","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/nn.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/nn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f51cf2cbd3e598d8"}},{"code_sha256_prefix":"cd23a508b079e2d4","entry":"gelu_jit","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/nn.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/nn.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd23a508b079e2d4"}},{"code_sha256_prefix":"141937afc98f8d1e","entry":"get_named_beta_schedule","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/gaussian_diffusion.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"141937afc98f8d1e"}},{"code_sha256_prefix":"f6d7c009a8efb8b7","entry":"mean_flat","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/nn.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/nn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6d7c009a8efb8b7"}},{"code_sha256_prefix":"c3bf3f13d5f63d5b","entry":"normal_kl","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/losses.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/losses.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3bf3f13d5f63d5b"}},{"code_sha256_prefix":"4e881da877b25beb","entry":"space_timesteps","repo":"deep-floyd/IF","repo_kind":"found_in_text","path":"deepfloyd_if/model/respace.py","file_url":"https://github.com/deep-floyd/IF/blob/HEAD/deepfloyd_if/model/respace.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e881da877b25beb"}},{"code_sha256_prefix":"88a5c5be88a2747b","entry":"AttentionBlock","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"88a5c5be88a2747b"}},{"code_sha256_prefix":"47ec7613fe517e5e","entry":"MOVQ","repo":"ai-forever/MoVQGAN","repo_kind":"found_in_text","path":"movqgan/models/vqgan.py","file_url":"https://github.com/ai-forever/MoVQGAN/blob/HEAD/movqgan/models/vqgan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"47ec7613fe517e5e"}},{"code_sha256_prefix":"99ce6c633e8c9151","entry":"ResBlock","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99ce6c633e8c9151"}},{"code_sha256_prefix":"7759535afd6da808","entry":"Text2ImUNet","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7759535afd6da808"}},{"code_sha256_prefix":"e587c8596212bcdf","entry":"TimestepEmbedSequential","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e587c8596212bcdf"}},{"code_sha256_prefix":"820bf30db90e06e6","entry":"UNetModel","repo":"ai-forever/Kandinsky-2","repo_kind":"official","path":"kandinsky2/model/text2im_model2_1.py","file_url":"https://github.com/ai-forever/Kandinsky-2/blob/HEAD/kandinsky2/model/text2im_model2_1.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"820bf30db90e06e6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}