{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/accelerating-auto-regressive-text-to-image","title":"Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding","arxiv_id":"2410.01699","date":"2024-10-02","proceeding":null,"authors":["Yao Teng","Han Shi","Xian Liu","Xuefei Ning","Guohao Dai","Yu Wang","Zhenguo Li","Xihui Liu"],"abstract":"The current large auto-regressive models can generate high-quality, high-resolution images, but these models require hundreds or even thousands of steps of next-token prediction during inference, resulting in substantial time consumption. In existing studies, Jacobi decoding, an iterative parallel decoding algorithm, has been used to accelerate the auto-regressive generation and can be executed without training. However, the Jacobi decoding relies on a deterministic criterion to determine the convergence of iterations. Thus, it works for greedy decoding but is incompatible with sampling-based decoding which is crucial for visual quality and diversity in the current auto-regressive text-to-image generation. In this paper, we propose a training-free probabilistic parallel decoding algorithm, Speculative Jacobi Decoding (SJD), to accelerate auto-regressive text-to-image generation. By introducing a probabilistic convergence criterion, our SJD accelerates the inference of auto-regressive text-to-image generation while maintaining the randomness in sampling-based token decoding and allowing the model to generate diverse images. Specifically, SJD facilitates the model to predict multiple tokens at each step and accepts tokens based on the probabilistic criterion, enabling the model to generate images with fewer steps than the conventional next-token-prediction paradigm. We also investigate the token initialization strategies that leverage the spatial locality of visual data to further improve the acceleration ratio under specific scenarios. We conduct experiments for our proposed SJD on multiple auto-regressive text-to-image generation models, showing the effectiveness of model acceleration without sacrificing the visual quality.","url_abs":"https://arxiv.org/abs/2410.01699v1","url_pdf":"https://arxiv.org/pdf/2410.01699v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"accelerating-auto-regressive-text-to-image","repo_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"accelerating-auto-regressive-text-to-image","repo_url":"https://github.com/wdrink/simplear","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.01699","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.01699"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wdrink/simplear","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_fixture":3,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"4ab810c163e93998","entry":"SpeculativeSampler","repo":"tyshiwo1/Accelerating-T2I-AR-with-SJD","repo_kind":"official","path":"scheduler/jacobi_iteration_lumina_mgpt.py","file_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD/blob/HEAD/scheduler/jacobi_iteration_lumina_mgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ab810c163e93998"}},{"code_sha256_prefix":"e551900d97d9e719","entry":"prefix_matching_next_tokens","repo":"tyshiwo1/Accelerating-T2I-AR-with-SJD","repo_kind":"official","path":"scheduler/jacobi_iteration_lumina_mgpt.py","file_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD/blob/HEAD/scheduler/jacobi_iteration_lumina_mgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e551900d97d9e719"}},{"code_sha256_prefix":"5971c96f289fd9d1","entry":"push_forward_model_kwargs_and_inputs","repo":"tyshiwo1/Accelerating-T2I-AR-with-SJD","repo_kind":"official","path":"scheduler/jacobi_iteration_lumina_mgpt.py","file_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD/blob/HEAD/scheduler/jacobi_iteration_lumina_mgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5971c96f289fd9d1"}},{"code_sha256_prefix":"9670c55f7a16bad4","entry":"sampling_logits2tokens","repo":"tyshiwo1/Accelerating-T2I-AR-with-SJD","repo_kind":"official","path":"scheduler/jacobi_iteration_lumina_mgpt.py","file_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD/blob/HEAD/scheduler/jacobi_iteration_lumina_mgpt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9670c55f7a16bad4"}},{"code_sha256_prefix":"32615dc76060033c","entry":"get_multi_token_for_preparation","repo":"tyshiwo1/Accelerating-T2I-AR-with-SJD","repo_kind":"official","path":"scheduler/jacobi_iteration_lumina_mgpt.py","file_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD/blob/HEAD/scheduler/jacobi_iteration_lumina_mgpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32615dc76060033c"}},{"code_sha256_prefix":"b0a6dcaa8515643f","entry":"renew_sampler","repo":"tyshiwo1/Accelerating-T2I-AR-with-SJD","repo_kind":"official","path":"scheduler/jacobi_iteration_lumina_mgpt.py","file_url":"https://github.com/tyshiwo1/Accelerating-T2I-AR-with-SJD/blob/HEAD/scheduler/jacobi_iteration_lumina_mgpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0a6dcaa8515643f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}