{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2512-14681","title":"Fast and Accurate Causal Parallel Decoding using Jacobi Forcing","arxiv_id":"2512.14681","date":"2025-12-16","proceeding":"ICML","authors":["Lanxiang Hu","Siqi Kou","Yichao Fu","Samyam Rajbhandari","Tajana Rosing","Yuxiong He","Zhijie Deng","Hao Zhang"],"abstract":"Multi-token generation has emerged as a promising paradigm for accelerating transformer-based large model inference. Recent efforts primarily explore diffusion Large Language Models (dLLMs) for parallel decoding to reduce inference latency. To achieve AR-level generation quality, many techniques adapt AR models into dLLMs to enable parallel decoding. However, they suffer from limited speedup compared to AR models due to a pretrain-to-posttrain mismatch. Specifically, the masked data distribution in post-training deviates significantly from the real-world data distribution seen during pretraining, and dLLMs rely on bidirectional attention, which conflicts with the causal prior learned during pretraining and hinders the integration of exact KV cache reuse. To address this, we introduce Jacobi Forcing, a progressive distillation paradigm where models are trained on their own generated parallel decoding trajectories, smoothly shifting AR models into efficient parallel decoders while preserving their pretrained causal inference property. The models trained under this paradigm, Jacobi Forcing Model, achieves 3.8x wall-clock speedup on coding and math benchmarks with minimal loss in performance. Based on Jacobi Forcing Models' trajectory characteristics, we introduce multi-block decoding with rejection recycling, which enables up to 4.5x higher token acceptance count per iteration and nearly 4.0x wall-clock speedup, effectively trading additional compute for lower inference latency. Our code is available at https://github.com/hao-ai-lab/JacobiForcing.","url_abs":"https://arxiv.org/abs/2512.14681","url_pdf":"https://arxiv.org/pdf/2512.14681","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2512.14681","atlas_url":"https://app.syntology.ai/?focus=2512.14681","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2512.14681"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/hao-ai-lab/JacobiForcing","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":7},"by_repo_kind":{"found_in_text":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f650a566b3a44b39","entry":"extract_python_code","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/ar_inference_baseline.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/ar_inference_baseline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f650a566b3a44b39"}},{"code_sha256_prefix":"f30e3943ccb923f5","entry":"extract_python_code","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/jacobi_forcing_inference_MR_humaneval.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/jacobi_forcing_inference_MR_humaneval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f30e3943ccb923f5"}},{"code_sha256_prefix":"935fb30ac04be7c7","entry":"extract_python_code","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/jacobi_forcing_inference_MR_humaneval_config_grid_search.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/jacobi_forcing_inference_MR_humaneval_config_grid_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"935fb30ac04be7c7"}},{"code_sha256_prefix":"163eb1381b9a92fb","entry":"load_jsonl","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/ar_inference_baseline.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/ar_inference_baseline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"163eb1381b9a92fb"}},{"code_sha256_prefix":"2083c5c3af717627","entry":"make_collate_fn","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/train/baseline_sft_train.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/train/baseline_sft_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2083c5c3af717627"}},{"code_sha256_prefix":"bd5ce784b2f6ab8b","entry":"make_jacobian_data_module","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/train/deprecated/train_soft_cllm_loss.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/train/deprecated/train_soft_cllm_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bd5ce784b2f6ab8b"}},{"code_sha256_prefix":"3849703bd564244d","entry":"preprocess_distill_data","repo":"hao-ai-lab/JacobiForcing","repo_kind":"found_in_text","path":"JacobiForcing/train/deprecated/train_soft_cllm_loss.py","file_url":"https://github.com/hao-ai-lab/JacobiForcing/blob/HEAD/JacobiForcing/train/deprecated/train_soft_cllm_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3849703bd564244d"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}