{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visual-autoregressive-modeling-scalable-image","title":"Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction","arxiv_id":"2404.02905","date":"2024-04-03","proceeding":null,"authors":["Keyu Tian","Yi Jiang","Zehuan Yuan","Bingyue Peng","LiWei Wang"],"abstract":"We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine \"next-scale prediction\" or \"next-resolution prediction\", diverging from the standard raster-scan \"next-token prediction\". This simple, intuitive methodology allows autoregressive (AR) transformers to learn visual distributions fast and generalize well: VAR, for the first time, makes GPT-like AR models surpass diffusion transformers in image generation. On ImageNet 256x256 benchmark, VAR significantly improve AR baseline by improving Frechet inception distance (FID) from 18.65 to 1.73, inception score (IS) from 80.4 to 350.2, with around 20x faster inference speed. It is also empirically verified that VAR outperforms the Diffusion Transformer (DiT) in multiple dimensions including image quality, inference speed, data efficiency, and scalability. Scaling up VAR models exhibits clear power-law scaling laws similar to those observed in LLMs, with linear correlation coefficients near -0.998 as solid evidence. VAR further showcases zero-shot generalization ability in downstream tasks including image in-painting, out-painting, and editing. These results suggest VAR has initially emulated the two important properties of LLMs: Scaling Laws and zero-shot task generalization. We have released all models and codes to promote the exploration of AR/VAR models for visual generation and unified learning.","url_abs":"https://arxiv.org/abs/2404.02905v2","url_pdf":"https://arxiv.org/pdf/2404.02905v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-autoregressive-modeling-scalable-image","repo_url":"https://github.com/FoundationVision/VAR","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"visual-autoregressive-modeling-scalable-image","repo_url":"https://github.com/EkaterinaXie/LiteVAR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"visual-autoregressive-modeling-scalable-image","repo_url":"https://github.com/FoundationVision/Infinity","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"adaptive-instance-normalization","method_name":"Adaptive Instance Normalization"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"gpt","method_name":"GPT"},{"method_slug":"gpt-2","method_name":"GPT-2"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vq-vae","method_name":"VQ-VAE"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-imagenet-256x256","task":"Image Generation","dataset":"ImageNet 256x256","model":"VAR (Visual Autoregressive)","rank_in_archive_order":37,"of":94,"metrics":{"FID":"1.73"},"uses_additional_data":false},{"leaderboard":"/sota/image-reconstruction-on-ultra-high-resolution","task":"Image Reconstruction","dataset":"Ultra-High Resolution Image Reconstruction Benchmark","model":"VAR (16x16)","rank_in_archive_order":6,"of":6,"metrics":{"PSNR":"21.79","rFID":"9.85"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.02905","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.02905"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/FoundationVision/Infinity","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/FoundationVision/VAR","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EkaterinaXie/LiteVAR","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_fixture":2,"unverified":6},"by_repo_kind":{"listed":{"samples":11,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9fcdaa6e423e8aa7","entry":"Normalize","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"models/basic_vae.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/models/basic_vae.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fcdaa6e423e8aa7"}},{"code_sha256_prefix":"6a19741de3a68058","entry":"get_params_num","repo":"FoundationVision/Infinity","repo_kind":"listed","path":"infinity/models/infinity.py","file_url":"https://github.com/FoundationVision/Infinity/blob/HEAD/infinity/models/infinity.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a19741de3a68058"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"models/basic_vae.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/models/basic_vae.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"9526ce09c7148703","entry":"sample_with_top_k_top_p_also_inplace_modifying_logits_","repo":"FoundationVision/Infinity","repo_kind":"listed","path":"infinity/models/infinity.py","file_url":"https://github.com/FoundationVision/Infinity/blob/HEAD/infinity/models/infinity.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9526ce09c7148703"}},{"code_sha256_prefix":"5451c3f14c7e520c","entry":"sampling_with_top_k_top_p_also_inplace_modifying_probs_","repo":"FoundationVision/Infinity","repo_kind":"listed","path":"infinity/models/infinity.py","file_url":"https://github.com/FoundationVision/Infinity/blob/HEAD/infinity/models/infinity.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5451c3f14c7e520c"}},{"code_sha256_prefix":"e847ee15f089dee6","entry":"allgather","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"dist.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/dist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e847ee15f089dee6"}},{"code_sha256_prefix":"ff4c548fd07140d0","entry":"allreduce","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"dist.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/dist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff4c548fd07140d0"}},{"code_sha256_prefix":"793c7e2489dd8dfc","entry":"make_attn","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"models/basic_vae.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/models/basic_vae.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"793c7e2489dd8dfc"}},{"code_sha256_prefix":"826ea3fc406a6a09","entry":"new_group","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"dist.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/dist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"826ea3fc406a6a09"}},{"code_sha256_prefix":"3c57a99a764d06f2","entry":"should_replace","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"demo_sample_batch_quant.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/demo_sample_batch_quant.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c57a99a764d06f2"}},{"code_sha256_prefix":"577a75b98b013aed","entry":"slow_attn_ban_mini_token_2","repo":"EkaterinaXie/LiteVAR","repo_kind":"listed","path":"models/ban_all_1.py","file_url":"https://github.com/EkaterinaXie/LiteVAR/blob/HEAD/models/ban_all_1.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"577a75b98b013aed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}