{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/specreason-fast-and-accurate-inference-time","title":"SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning","arxiv_id":"2504.07891","date":"2025-04-10","proceeding":null,"authors":["Rui Pan","Yinwei Dai","Zhihao Zhang","Gabriele Oliaro","Zhihao Jia","Ravi Netravali"],"abstract":"Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs). However, this improved accuracy comes at the cost of high inference latency due to the length of generated reasoning sequences and the autoregressive nature of decoding. Our key insight in tackling these overheads is that LRM inference, and the reasoning that it embeds, is highly tolerant of approximations: complex tasks are typically broken down into simpler steps, each of which brings utility based on the semantic insight it provides for downstream steps rather than the exact tokens it generates. Accordingly, we introduce SpecReason, a system that automatically accelerates LRM inference by using a lightweight model to (speculatively) carry out simpler intermediate reasoning steps and reserving the costly base model only to assess (and potentially correct) the speculated outputs. Importantly, SpecReason's focus on exploiting the semantic flexibility of thinking tokens in preserving final-answer accuracy is complementary to prior speculation techniques, most notably speculative decoding, which demands token-level equivalence at each step. Across a variety of reasoning benchmarks, SpecReason achieves 1.5-2.5$\\times$ speedup over vanilla LRM inference while improving accuracy by 1.0-9.9\\%. Compared to speculative decoding without SpecReason, their combination yields an additional 19.4-44.2\\% latency reduction. We open-source SpecReason at https://github.com/ruipeterpan/specreason.","url_abs":"https://arxiv.org/abs/2504.07891v1","url_pdf":"https://arxiv.org/pdf/2504.07891v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"specreason-fast-and-accurate-inference-time","repo_url":"https://github.com/ruipeterpan/specreason","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.07891","atlas_url":"https://app.syntology.ai/?focus=2504.07891","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.07891"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruipeterpan/specreason","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"2c877d8ef01c058c","entry":"get_avg_score","repo":"ruipeterpan/specreason","repo_kind":"official","path":"spec_reason.py","file_url":"https://github.com/ruipeterpan/specreason/blob/HEAD/spec_reason.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c877d8ef01c058c"}},{"code_sha256_prefix":"8cc6f4ebef56a6b2","entry":"get_frequency","repo":"ruipeterpan/specreason","repo_kind":"official","path":"spec_reason.py","file_url":"https://github.com/ruipeterpan/specreason/blob/HEAD/spec_reason.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8cc6f4ebef56a6b2"}},{"code_sha256_prefix":"7172629cfa1a5431","entry":"get_model","repo":"ruipeterpan/specreason","repo_kind":"official","path":"spec_reason.py","file_url":"https://github.com/ruipeterpan/specreason/blob/HEAD/spec_reason.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7172629cfa1a5431"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}