{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaling-autoregressive-models-for-content","title":"Scaling Autoregressive Models for Content-Rich Text-to-Image Generation","arxiv_id":"2206.10789","date":"2022-06-22","proceeding":null,"authors":["Jiahui Yu","Yuanzhong Xu","Jing Yu Koh","Thang Luong","Gunjan Baid","ZiRui Wang","Vijay Vasudevan","Alexander Ku","Yinfei Yang","Burcu Karagol Ayan","Ben Hutchinson","Wei Han","Zarana Parekh","Xin Li","Han Zhang","Jason Baldridge","Yonghui Wu"],"abstract":"We present the Pathways Autoregressive Text-to-Image (Parti) model, which generates high-fidelity photorealistic images and supports content-rich synthesis involving complex compositions and world knowledge. Parti treats text-to-image generation as a sequence-to-sequence modeling problem, akin to machine translation, with sequences of image tokens as the target outputs rather than text tokens in another language. This strategy can naturally tap into the rich body of prior work on large language models, which have seen continued advances in capabilities and performance through scaling data and model sizes. Our approach is simple: First, Parti uses a Transformer-based image tokenizer, ViT-VQGAN, to encode images as sequences of discrete tokens. Second, we achieve consistent quality improvements by scaling the encoder-decoder Transformer model up to 20B parameters, with a new state-of-the-art zero-shot FID score of 7.23 and finetuned FID score of 3.22 on MS-COCO. Our detailed analysis on Localized Narratives as well as PartiPrompts (P2), a new holistic benchmark of over 1600 English prompts, demonstrate the effectiveness of Parti across a wide variety of categories and difficulty aspects. We also explore and highlight limitations of our models in order to define and exemplify key areas of focus for further improvements. See https://parti.research.google/ for high-resolution images.","url_abs":"https://arxiv.org/abs/2206.10789v1","url_pdf":"https://arxiv.org/pdf/2206.10789v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaling-autoregressive-models-for-content","repo_url":"https://github.com/lucidrains/parti-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"scaling-autoregressive-models-for-content","repo_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"},{"task_slug":"world-knowledge","task_name":"World Knowledge"},{"task_slug":"2017-2022-1-7-3940","task_name":"陕西、四川、重庆地区2017-2022年末人口数（如图1-7所示）中，陕西省人口数较稳 定，维持在3940万人，四川和重庆均呈上升趋势。"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[{"slug":"p2","name":"P2","full_name":"PartiPrompts"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/text-to-image-generation-on-coco-1","task":"Text-to-Image Generation","dataset":"COCO","model":"Parti Finetuned","rank_in_archive_order":1,"of":3,"metrics":{"FID":"3.22"},"uses_additional_data":false},{"leaderboard":"/sota/text-to-image-generation-on-coco-1","task":"Text-to-Image Generation","dataset":"COCO","model":"Parti","rank_in_archive_order":3,"of":3,"metrics":{"FID":"7.23"},"uses_additional_data":false},{"leaderboard":"/sota/text-to-image-generation-on-laion-coco","task":"Text-to-Image Generation","dataset":"LAION COCO","model":"Parti Finetuned","rank_in_archive_order":1,"of":2,"metrics":{"FID":"8.39"},"uses_additional_data":false},{"leaderboard":"/sota/text-to-image-generation-on-laion-coco","task":"Text-to-Image Generation","dataset":"LAION COCO","model":"Parti","rank_in_archive_order":2,"of":2,"metrics":{"FID":"15.97"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.10789","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.10789"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucidrains/parti-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":2,"ran_draft_wrong":1,"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"c16aee9490eb8729","entry":"eval_decorator","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c16aee9490eb8729"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"5e9bf34da126e231","entry":"basic_clean","repo":"syang-lab/Pathway_Autoregressive_Text2Image_Model","repo_kind":"listed","path":"Parti_Pytorch_V5/parti/utils/tokenizer.py","file_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model/blob/HEAD/Parti_Pytorch_V5/parti/utils/tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e9bf34da126e231"}},{"code_sha256_prefix":"34120eaab3037c24","entry":"get_ckpt_path","repo":"syang-lab/Pathway_Autoregressive_Text2Image_Model","repo_kind":"listed","path":"Parti_Pytorch_V5/parti/losses/lpips.py","file_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model/blob/HEAD/Parti_Pytorch_V5/parti/losses/lpips.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34120eaab3037c24"}},{"code_sha256_prefix":"4ab4f9928f106ac4","entry":"get_pairs","repo":"syang-lab/Pathway_Autoregressive_Text2Image_Model","repo_kind":"listed","path":"Parti_Pytorch_V5/parti/utils/tokenizer.py","file_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model/blob/HEAD/Parti_Pytorch_V5/parti/utils/tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ab4f9928f106ac4"}},{"code_sha256_prefix":"7b13ea17a4f124b2","entry":"norm_tensor","repo":"syang-lab/Pathway_Autoregressive_Text2Image_Model","repo_kind":"listed","path":"Parti_Pytorch_V5/parti/losses/lpips.py","file_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model/blob/HEAD/Parti_Pytorch_V5/parti/losses/lpips.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b13ea17a4f124b2"}},{"code_sha256_prefix":"8afa5135930de8c8","entry":"spatial_average","repo":"syang-lab/Pathway_Autoregressive_Text2Image_Model","repo_kind":"listed","path":"Parti_Pytorch_V5/parti/losses/lpips.py","file_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model/blob/HEAD/Parti_Pytorch_V5/parti/losses/lpips.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8afa5135930de8c8"}},{"code_sha256_prefix":"4edd24da6117e968","entry":"whitespace_clean","repo":"syang-lab/Pathway_Autoregressive_Text2Image_Model","repo_kind":"listed","path":"Parti_Pytorch_V5/parti/utils/tokenizer.py","file_url":"https://github.com/syang-lab/Pathway_Autoregressive_Text2Image_Model/blob/HEAD/Parti_Pytorch_V5/parti/utils/tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4edd24da6117e968"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}