{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/playground-v3-improving-text-to-image","title":"Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models","arxiv_id":"2409.10695","date":"2024-09-16","proceeding":null,"authors":["Bingchen Liu","Ehsan Akhgari","Alexander Visheratin","Aleks Kamko","Linmiao Xu","Shivam Shrirao","Chase Lambert","Joao Souza","Suhail Doshi","Daiqing Li"],"abstract":"We introduce Playground v3 (PGv3), our latest text-to-image model that achieves state-of-the-art (SoTA) performance across multiple testing benchmarks, excels in graphic design abilities and introduces new capabilities. Unlike traditional text-to-image generative models that rely on pre-trained language models like T5 or CLIP text encoders, our approach fully integrates Large Language Models (LLMs) with a novel structure that leverages text conditions exclusively from a decoder-only LLM. Additionally, to enhance image captioning quality-we developed an in-house captioner, capable of generating captions with varying levels of detail, enriching the diversity of text structures. We also introduce a new benchmark CapsBench to evaluate detailed image captioning performance. Experimental results demonstrate that PGv3 excels in text prompt adherence, complex reasoning, and accurate text rendering. User preference studies indicate the super-human graphic design ability of our model for common design applications, such as stickers, posters, and logo designs. Furthermore, PGv3 introduces new capabilities, including precise RGB color control and robust multilingual understanding.","url_abs":"https://arxiv.org/abs/2409.10695v2","url_pdf":"https://arxiv.org/pdf/2409.10695v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"playground-v3-improving-text-to-image","repo_url":"https://github.com/yuchuantian/u-dit","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"image-captioning","task_name":"Image Captioning"}],"methods":[{"method_slug":"adafactor","method_name":"Adafactor"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"glu","method_name":"Gated Linear Unit"},{"method_slug":"inverse-square-root-schedule","method_name":"Inverse Square Root Schedule"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sentencepiece","method_name":"SentencePiece"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"t5","method_name":"T5"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.10695","atlas_url":"https://app.syntology.ai/?focus=2409.10695","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.10695"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuchuantian/u-dit","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":7,"ran_honours":4,"ran_violates":1,"ran_fixture":2,"ran":3,"unverified":4},"by_repo_kind":{"listed":{"samples":21,"ran":17,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":21,"samples":[{"code_sha256_prefix":"d6a68e210556f857","entry":"approx_standard_normal_cdf","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d6a68e210556f857"}},{"code_sha256_prefix":"1712a07966b542ee","entry":"center_crop_arr","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"train_accelerate.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/train_accelerate.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1712a07966b542ee"}},{"code_sha256_prefix":"ab1c9568b4e13899","entry":"continuous_gaussian_log_likelihood","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab1c9568b4e13899"}},{"code_sha256_prefix":"b2cc78f4103df806","entry":"create_logger","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"train_accelerate.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/train_accelerate.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2cc78f4103df806"}},{"code_sha256_prefix":"7b21a01ae77703a3","entry":"create_npz_from_sample_folder","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"sample_ddp.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/sample_ddp.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b21a01ae77703a3"}},{"code_sha256_prefix":"c92c27c924b517e8","entry":"get_2d_sincos_pos_embed","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"models.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c92c27c924b517e8"}},{"code_sha256_prefix":"3e0fa4efc22272d4","entry":"get_beta_schedule","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e0fa4efc22272d4"}},{"code_sha256_prefix":"36e30c7fb679ec78","entry":"get_named_beta_schedule","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36e30c7fb679ec78"}},{"code_sha256_prefix":"f6d7c009a8efb8b7","entry":"mean_flat","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6d7c009a8efb8b7"}},{"code_sha256_prefix":"03310bba324ae4fb","entry":"modulate","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"models.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03310bba324ae4fb"}},{"code_sha256_prefix":"8afbfc42c6ea0448","entry":"normal_kl","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8afbfc42c6ea0448"}},{"code_sha256_prefix":"f55137c53a3a4557","entry":"precompute_freqs_cis_2d","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"udit_models.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/udit_models.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f55137c53a3a4557"}},{"code_sha256_prefix":"20ff6d11dd593018","entry":"reshape_for_broadcast","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"udit_models.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/udit_models.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"20ff6d11dd593018"}},{"code_sha256_prefix":"ea9dbc131adf582e","entry":"space_timesteps","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/respace.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/respace.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ea9dbc131adf582e"}},{"code_sha256_prefix":"dc19b15524ec8454","entry":"trunc_normal_","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"udit_models.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/udit_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"dc19b15524ec8454"}},{"code_sha256_prefix":"5b36e74896cd9928","entry":"window_partition","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"utils/udit_models_original.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/utils/udit_models_original.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5b36e74896cd9928"}},{"code_sha256_prefix":"800dfbdedee84978","entry":"window_reverse","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"utils/udit_models_original.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/utils/udit_models_original.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"800dfbdedee84978"}},{"code_sha256_prefix":"e48218d7d73db0b3","entry":"create_named_schedule_sampler","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"diffusion/timestep_sampler.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/diffusion/timestep_sampler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e48218d7d73db0b3"}},{"code_sha256_prefix":"6a0d5ecd441905cc","entry":"download_model","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"download.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/download.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a0d5ecd441905cc"}},{"code_sha256_prefix":"29947a0a94157558","entry":"find_model","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"download.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/download.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"29947a0a94157558"}},{"code_sha256_prefix":"665d8a4e8f673a4c","entry":"get_2d_sincos_pos_embed_from_grid","repo":"yuchuantian/u-dit","repo_kind":"listed","path":"models.py","file_url":"https://github.com/yuchuantian/u-dit/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"665d8a4e8f673a4c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}