{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tokenskip-controllable-chain-of-thought","title":"TokenSkip: Controllable Chain-of-Thought Compression in LLMs","arxiv_id":"2502.12067","date":"2025-02-17","proceeding":null,"authors":["Heming Xia","Yongqi Li","Chak Tou Leong","Wenjie Wang","Wenjie Li"],"abstract":"Chain-of-Thought (CoT) has been proven effective in enhancing the reasoning capabilities of large language models (LLMs). Recent advancements, such as OpenAI's o1 and DeepSeek-R1, suggest that scaling up the length of CoT sequences during inference could further boost LLM reasoning performance. However, due to the autoregressive nature of LLM decoding, longer CoT outputs lead to a linear increase in inference latency, adversely affecting user experience, particularly when the CoT exceeds 10,000 tokens. To address this limitation, we analyze the semantic importance of tokens within CoT outputs and reveal that their contributions to reasoning vary. Building on this insight, we propose TokenSkip, a simple yet effective approach that enables LLMs to selectively skip less important tokens, allowing for controllable CoT compression. Extensive experiments across various models and tasks demonstrate the effectiveness of TokenSkip in reducing CoT token usage while preserving strong reasoning performance. Notably, when applied to Qwen2.5-14B-Instruct, TokenSkip reduces reasoning tokens by 40% (from 313 to 181) on GSM8K, with less than a 0.4% performance drop.","url_abs":"https://arxiv.org/abs/2502.12067v1","url_pdf":"https://arxiv.org/pdf/2502.12067v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tokenskip-controllable-chain-of-thought","repo_url":"https://github.com/hemingkx/tokenskip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.12067","atlas_url":"https://app.syntology.ai/?focus=2502.12067","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.12067"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hemingkx/tokenskip","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/hemingkx/TokenSkip","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":7,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6390dea763ce122e","entry":"extract_boxed_answers","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"data_processing/answer_extraction.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/data_processing/answer_extraction.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6390dea763ce122e"}},{"code_sha256_prefix":"0772e5020f6b0849","entry":"extract_program_output","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"data_processing/answer_extraction.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/data_processing/answer_extraction.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0772e5020f6b0849"}},{"code_sha256_prefix":"14ec4b04b4360475","entry":"generate_completions","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/eval/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14ec4b04b4360475"}},{"code_sha256_prefix":"1f5ed2063eb79f0c","entry":"load_all_data","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"get_llamafactory_input.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/get_llamafactory_input.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f5ed2063eb79f0c"}},{"code_sha256_prefix":"57518f0194aae8fb","entry":"load_json","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"get_llamafactory_input.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/get_llamafactory_input.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"57518f0194aae8fb"}},{"code_sha256_prefix":"3dc03e3fd9c53b25","entry":"load_jsonl","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"LLMLingua.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/LLMLingua.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3dc03e3fd9c53b25"}},{"code_sha256_prefix":"5abed389274c4c03","entry":"strip_string","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"data_processing/answer_extraction.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/data_processing/answer_extraction.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5abed389274c4c03"}},{"code_sha256_prefix":"1d961161c71cc153","entry":"extract_program","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"eval/eval_utils.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/eval/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d961161c71cc153"}},{"code_sha256_prefix":"19dce514835bd816","entry":"get_next_word_predictions","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"19dce514835bd816"}},{"code_sha256_prefix":"49e328786ffb3f8f","entry":"parse_question","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"eval/eval_utils.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/eval/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"49e328786ffb3f8f"}},{"code_sha256_prefix":"3deb39de933b985a","entry":"read_data","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"evaluation.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3deb39de933b985a"}},{"code_sha256_prefix":"8c63844c66973f8d","entry":"score_completions","repo":"hemingkx/TokenSkip","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/hemingkx/TokenSkip/blob/HEAD/eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c63844c66973f8d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}