{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-20342","title":"Teaching Language Models to Reason with Tools","arxiv_id":"2510.20342","date":"2025-10-23","proceeding":null,"authors":["Chengpeng Li","Zhengyang Tang","Ziniu Li","Mingfeng Xue","Keqin Bao","Tian Ding","Ruoyu Sun","Benyou Wang","Xiang Wang","Junyang Lin","Dayiheng Liu"],"abstract":"Large reasoning models (LRMs) like OpenAI-o1 have shown impressive capabilities in natural language reasoning. However, these models frequently demonstrate inefficiencies or inaccuracies when tackling complex mathematical operations. While integrating computational tools such as Code Interpreters (CIs) offers a promising solution, it introduces a critical challenge: a conflict between the model's internal, probabilistic reasoning and the external, deterministic knowledge provided by the CI, which often leads models to unproductive deliberation. To overcome this, we introduce CoRT (Code-Optimized Reasoning Training), a post-training framework designed to teach LRMs to effectively utilize CIs. We propose \\emph{Hint-Engineering}, a new data synthesis strategy that strategically injects diverse hints at optimal points within reasoning paths. This approach generates high-quality, code-integrated reasoning data specifically tailored to optimize LRM-CI interaction. Using this method, we have synthesized 30 high-quality samples to post-train models ranging from 1.5B to 32B parameters through supervised fine-tuning. CoRT further refines the multi-round interleaving of external CI usage and internal thinking by employing rejection sampling and reinforcement learning. Our experimental evaluations demonstrate CoRT's effectiveness, yielding absolute improvements of 4\\% and 8\\% on DeepSeek-R1-Distill-Qwen-32B and DeepSeek-R1-Distill-Qwen-1.5B, respectively, across five challenging mathematical reasoning datasets. Moreover, CoRT significantly enhances efficiency, reducing token usage by approximately 30\\% for the 32B model and 50\\% for the 1.5B model compared to pure natural language reasoning baselines. The models and code are available at: https://github.com/ChengpengLi1003/CoRT.","url_abs":"https://arxiv.org/abs/2510.20342","url_pdf":"https://arxiv.org/pdf/2510.20342","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2510.20342","atlas_url":"https://app.syntology.ai/?focus=2510.20342","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.20342"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ChengpengLi1003/CoRT","reach":{"status":"ok"}}],"summary":{"ran":1,"unverified":10},"by_repo_kind":{"found_in_text":{"samples":11,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"0af0b06d6cf22524","entry":"lower_keys","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/utils.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0af0b06d6cf22524"}},{"code_sha256_prefix":"197bafc8e36a5af9","entry":"all_error_output","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"deepscaler/deepscaler/rewards/math_reward_hybrid.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/deepscaler/deepscaler/rewards/math_reward_hybrid.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"197bafc8e36a5af9"}},{"code_sha256_prefix":"7d89975b4585e1eb","entry":"calculate_sequence_entropy","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"evaluation/compute_metrics_dp.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/evaluation/compute_metrics_dp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d89975b4585e1eb"}},{"code_sha256_prefix":"15fcb6b739c07dac","entry":"call_oai_grm","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"deepscaler/deepscaler/utils.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/deepscaler/deepscaler/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15fcb6b739c07dac"}},{"code_sha256_prefix":"ecdacf069c9d10db","entry":"construct_prompt","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/utils.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ecdacf069c9d10db"}},{"code_sha256_prefix":"951c55f8667c735f","entry":"extract_answer","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/parser.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/parser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"951c55f8667c735f"}},{"code_sha256_prefix":"1d961161c71cc153","entry":"extract_program","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/parser.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/parser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1d961161c71cc153"}},{"code_sha256_prefix":"d71007c81129b37f","entry":"extract_programs_and_outputs","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"deepscaler/deepscaler/rewards/math_reward_hybrid.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/deepscaler/deepscaler/rewards/math_reward_hybrid.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d71007c81129b37f"}},{"code_sha256_prefix":"3bb02b5083de68fd","entry":"load_prompt","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/utils.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3bb02b5083de68fd"}},{"code_sha256_prefix":"882dab4c9c62e8c0","entry":"prepare_data","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/inference_vllm_dp_mj.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/inference_vllm_dp_mj.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"882dab4c9c62e8c0"}},{"code_sha256_prefix":"06b04cb075356fb9","entry":"strip_string","repo":"ChengpengLi1003/CoRT","repo_kind":"found_in_text","path":"infer/parser.py","file_url":"https://github.com/ChengpengLi1003/CoRT/blob/HEAD/infer/parser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06b04cb075356fb9"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}