{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/d-ar-diffusion-via-autoregressive-models","title":"D-AR: Diffusion via Autoregressive Models","arxiv_id":"2505.23660","date":"2025-05-29","proceeding":null,"authors":["Ziteng Gao","Mike Zheng Shou"],"abstract":"This paper presents Diffusion via Autoregressive models (D-AR), a new paradigm recasting the image diffusion process as a vanilla autoregressive procedure in the standard next-token-prediction fashion. We start by designing the tokenizer that converts images into sequences of discrete tokens, where tokens in different positions can be decoded into different diffusion denoising steps in the pixel space. Thanks to the diffusion properties, these tokens naturally follow a coarse-to-fine order, which directly lends itself to autoregressive modeling. Therefore, we apply standard next-token prediction on these tokens, without modifying any underlying designs (either causal masks or training/inference strategies), and such sequential autoregressive token generation directly mirrors the diffusion procedure in image space. That is, once the autoregressive model generates an increment of tokens, we can directly decode these tokens into the corresponding diffusion denoising step in the streaming manner. Our pipeline naturally reveals several intriguing properties, for example, it supports consistent previews when generating only a subset of tokens and enables zero-shot layout-controlled synthesis. On the standard ImageNet benchmark, our method achieves 2.09 FID using a 775M Llama backbone with 256 discrete tokens. We hope our work can inspire future research on unified autoregressive architectures of visual synthesis, especially with large language models. Code and models will be available at https://github.com/showlab/D-AR","url_abs":"https://arxiv.org/abs/2505.23660v1","url_pdf":"https://arxiv.org/pdf/2505.23660v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"d-ar-diffusion-via-autoregressive-models","repo_url":"https://github.com/showlab/d-ar","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"llama","method_name":"LLaMA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.23660","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.23660"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/showlab/d-ar","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran_honours":2,"unverified":11},"by_repo_kind":{"official":{"samples":14,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1712a07966b542ee","entry":"center_crop_arr","repo":"showlab/d-ar","repo_kind":"official","path":"dataset/augmentation.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/dataset/augmentation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1712a07966b542ee"}},{"code_sha256_prefix":"f6ff7671338c9c92","entry":"find_multiple","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/models/gpt.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/models/gpt.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6ff7671338c9c92"}},{"code_sha256_prefix":"7f53b2cdcd16fad9","entry":"precompute_freqs_cis_2d","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/serve/gpt_model.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/serve/gpt_model.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f53b2cdcd16fad9"}},{"code_sha256_prefix":"5cceaa9d0edbce06","entry":"build_imagenet","repo":"showlab/d-ar","repo_kind":"official","path":"dataset/imagenet.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/dataset/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cceaa9d0edbce06"}},{"code_sha256_prefix":"b6635606f35f6a1a","entry":"build_imagenet_code","repo":"showlab/d-ar","repo_kind":"official","path":"dataset/imagenet.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/dataset/imagenet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b6635606f35f6a1a"}},{"code_sha256_prefix":"f0846b326b0d4987","entry":"decode_img_tensor","repo":"showlab/d-ar","repo_kind":"official","path":"sample_tokenizer.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/sample_tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0846b326b0d4987"}},{"code_sha256_prefix":"9d5534fb9e2589da","entry":"get_indices","repo":"showlab/d-ar","repo_kind":"official","path":"sample_tokenizer.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/sample_tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d5534fb9e2589da"}},{"code_sha256_prefix":"e8b5a038915ac61e","entry":"getitem","repo":"showlab/d-ar","repo_kind":"official","path":"config_util.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/config_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8b5a038915ac61e"}},{"code_sha256_prefix":"9dac59261197935f","entry":"logits_to_probs","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/models/generate.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/models/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9dac59261197935f"}},{"code_sha256_prefix":"0f0ff4e443413018","entry":"precompute_freqs_cis","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/models/gpt.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/models/gpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f0ff4e443413018"}},{"code_sha256_prefix":"aab87c3310945af0","entry":"precompute_freqs_cis_1d","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/models/gpt.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/models/gpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aab87c3310945af0"}},{"code_sha256_prefix":"980fb099f82d8b84","entry":"random_crop_arr","repo":"showlab/d-ar","repo_kind":"official","path":"dataset/augmentation.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/dataset/augmentation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"980fb099f82d8b84"}},{"code_sha256_prefix":"de8908ac0f302419","entry":"sample","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/models/generate.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/models/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de8908ac0f302419"}},{"code_sha256_prefix":"f2fea48028c7ae80","entry":"top_k_top_p_filtering","repo":"showlab/d-ar","repo_kind":"official","path":"autoregressive/models/generate.py","file_url":"https://github.com/showlab/d-ar/blob/HEAD/autoregressive/models/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2fea48028c7ae80"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}