{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autoregressive-model-beats-diffusion-llama","title":"Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation","arxiv_id":"2406.06525","date":"2024-06-10","proceeding":null,"authors":["Peize Sun","Yi Jiang","Shoufa Chen","Shilong Zhang","Bingyue Peng","Ping Luo","Zehuan Yuan"],"abstract":"We introduce LlamaGen, a new family of image generation models that apply original ``next-token prediction'' paradigm of large language models to visual generation domain. It is an affirmative answer to whether vanilla autoregressive models, e.g., Llama, without inductive biases on visual signals can achieve state-of-the-art image generation performance if scaling properly. We reexamine design spaces of image tokenizers, scalability properties of image generation models, and their training data quality. The outcome of this exploration consists of: (1) An image tokenizer with downsample ratio of 16, reconstruction quality of 0.94 rFID and codebook usage of 97% on ImageNet benchmark. (2) A series of class-conditional image generation models ranging from 111M to 3.1B parameters, achieving 2.18 FID on ImageNet 256x256 benchmarks, outperforming the popular diffusion models such as LDM, DiT. (3) A text-conditional image generation model with 775M parameters, from two-stage training on LAION-COCO and high aesthetics quality images, demonstrating competitive performance of visual quality and text alignment. (4) We verify the effectiveness of LLM serving frameworks in optimizing the inference speed of image generation models and achieve 326% - 414% speedup. We release all models and codes to facilitate open-source community of visual generation and multimodal foundation models.","url_abs":"https://arxiv.org/abs/2406.06525v1","url_pdf":"https://arxiv.org/pdf/2406.06525v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autoregressive-model-beats-diffusion-llama","repo_url":"https://github.com/foundationvision/llamagen","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"autoregressive-model-beats-diffusion-llama","repo_url":"https://github.com/0606zt/panollama","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"conditional-image-generation","task_name":"Conditional Image Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-imagenet-256x256","task":"Image Generation","dataset":"ImageNet 256x256","model":"LlamaGen","rank_in_archive_order":53,"of":94,"metrics":{"FID":"2.18"},"uses_additional_data":false},{"leaderboard":"/sota/image-reconstruction-on-ultra-high-resolution","task":"Image Reconstruction","dataset":"Ultra-High Resolution Image Reconstruction Benchmark","model":"LlamaGen (16x16)","rank_in_archive_order":4,"of":6,"metrics":{"LPIPS":"0.177","PSNR":"23.90","SSIM":"0.720","rFID":"5.59"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.06525","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.06525"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/foundationvision/llamagen","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/0606zt/panollama","reach":{"status":"ok"}}],"summary":{"ran":3,"ran_draft_wrong":1,"ran_honours":2,"unverified":9},"by_repo_kind":{"official":{"samples":15,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"02cb09299c18ee5d","entry":"build_t2i","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/t2i.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/t2i.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02cb09299c18ee5d"}},{"code_sha256_prefix":"ff9ed4675cb307aa","entry":"build_t2i_code","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/t2i.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/t2i.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff9ed4675cb307aa"}},{"code_sha256_prefix":"090d7b6b1260114e","entry":"build_t2i_image","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/t2i.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/t2i.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"090d7b6b1260114e"}},{"code_sha256_prefix":"1712a07966b542ee","entry":"center_crop_arr","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/augmentation.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/augmentation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1712a07966b542ee"}},{"code_sha256_prefix":"f6ff7671338c9c92","entry":"find_multiple","repo":"foundationvision/llamagen","repo_kind":"official","path":"autoregressive/models/gpt.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/autoregressive/models/gpt.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6ff7671338c9c92"}},{"code_sha256_prefix":"7f53b2cdcd16fad9","entry":"precompute_freqs_cis_2d","repo":"foundationvision/llamagen","repo_kind":"official","path":"autoregressive/models/gpt.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/autoregressive/models/gpt.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f53b2cdcd16fad9"}},{"code_sha256_prefix":"a5dc3fcd09ebc843","entry":"build_coco","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/coco.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/coco.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5dc3fcd09ebc843"}},{"code_sha256_prefix":"5cceaa9d0edbce06","entry":"build_imagenet","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/imagenet.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cceaa9d0edbce06"}},{"code_sha256_prefix":"b6635606f35f6a1a","entry":"build_imagenet_code","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/imagenet.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6635606f35f6a1a"}},{"code_sha256_prefix":"0a65aaf586d4f90b","entry":"build_openimage","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/openimage.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/openimage.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a65aaf586d4f90b"}},{"code_sha256_prefix":"9dac59261197935f","entry":"logits_to_probs","repo":"foundationvision/llamagen","repo_kind":"official","path":"autoregressive/models/generate.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/autoregressive/models/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9dac59261197935f"}},{"code_sha256_prefix":"0f0ff4e443413018","entry":"precompute_freqs_cis","repo":"foundationvision/llamagen","repo_kind":"official","path":"autoregressive/models/gpt.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/autoregressive/models/gpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f0ff4e443413018"}},{"code_sha256_prefix":"980fb099f82d8b84","entry":"random_crop_arr","repo":"foundationvision/llamagen","repo_kind":"official","path":"dataset/augmentation.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/dataset/augmentation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"980fb099f82d8b84"}},{"code_sha256_prefix":"de8908ac0f302419","entry":"sample","repo":"foundationvision/llamagen","repo_kind":"official","path":"autoregressive/models/generate.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/autoregressive/models/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de8908ac0f302419"}},{"code_sha256_prefix":"f2fea48028c7ae80","entry":"top_k_top_p_filtering","repo":"foundationvision/llamagen","repo_kind":"official","path":"autoregressive/models/generate.py","file_url":"https://github.com/foundationvision/llamagen/blob/HEAD/autoregressive/models/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2fea48028c7ae80"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}