{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-reasoning-for-llms-through","title":"Efficient Reasoning for LLMs through Speculative Chain-of-Thought","arxiv_id":"2504.19095","date":"2025-04-27","proceeding":null,"authors":["Jikai Wang","Juntao Li","Lijun Wu","Min Zhang"],"abstract":"Large reasoning language models such as OpenAI-o1 and Deepseek-R1 have recently attracted widespread attention due to their impressive task-solving abilities. However, the enormous model size and the generation of lengthy thought chains introduce significant reasoning costs and response latency. Existing methods for efficient reasoning mainly focus on reducing the number of model parameters or shortening the chain-of-thought length. In this paper, we introduce Speculative Chain-of-Thought (SCoT), which reduces reasoning latency from another perspective by accelerated average reasoning speed through large and small model collaboration. SCoT conducts thought-level drafting using a lightweight draft model. Then it selects the best CoT draft and corrects the error cases with the target model. The proposed thinking behavior alignment improves the efficiency of drafting and the draft selection strategy maintains the prediction accuracy for complex problems. Experimental results on GSM8K, MATH, GaoKao, CollegeMath and Olympiad datasets show that SCoT reduces reasoning latency by 48\\%$\\sim$66\\% for Deepseek-R1-Distill-Qwen-32B while achieving near-target-model-level performance. Our code is available at https://github.com/Jikai0Wang/Speculative_CoT.","url_abs":"https://arxiv.org/abs/2504.19095v1","url_pdf":"https://arxiv.org/pdf/2504.19095v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-reasoning-for-llms-through","repo_url":"https://github.com/jikai0wang/speculative_cot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2504.19095","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.19095"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jikai0wang/speculative_cot","reach":null}],"summary":{"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"3307a5046aa6d9ed","entry":"draft_cot","repo":"jikai0wang/speculative_cot","repo_kind":"official","path":"speculative_cot.py","file_url":"https://github.com/jikai0wang/speculative_cot/blob/HEAD/speculative_cot.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3307a5046aa6d9ed"}},{"code_sha256_prefix":"1bd40ab615e471b3","entry":"get_output","repo":"jikai0wang/speculative_cot","repo_kind":"official","path":"speculative_cot.py","file_url":"https://github.com/jikai0wang/speculative_cot/blob/HEAD/speculative_cot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bd40ab615e471b3"}},{"code_sha256_prefix":"d2e0b86d9fe7aea3","entry":"pick_draft","repo":"jikai0wang/speculative_cot","repo_kind":"official","path":"speculative_cot.py","file_url":"https://github.com/jikai0wang/speculative_cot/blob/HEAD/speculative_cot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d2e0b86d9fe7aea3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}