{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/research-learning-to-reason-with-search-for","title":"ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning","arxiv_id":"2503.19470","date":"2025-03-25","proceeding":null,"authors":["Mingyang Chen","Tianpeng Li","Haoze Sun","Yijie Zhou","Chenzheng Zhu","Haofen Wang","Jeff Z. Pan","Wen Zhang","Huajun Chen","Fan Yang","Zenan Zhou","WeiPeng Chen"],"abstract":"Large Language Models (LLMs) have shown remarkable capabilities in reasoning, exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating reasoning with external search processes remains challenging, especially for complex multi-hop questions requiring multiple retrieval steps. We propose ReSearch, a novel framework that trains LLMs to Reason with Search via reinforcement learning without using any supervised data on reasoning steps. Our approach treats search operations as integral components of the reasoning chain, where when and how to perform searches is guided by text-based thinking, and search results subsequently influence further reasoning. We train ReSearch on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct extensive experiments. Despite being trained on only one dataset, our models demonstrate strong generalizability across various benchmarks. Analysis reveals that ReSearch naturally elicits advanced reasoning capabilities such as reflection and self-correction during the reinforcement learning process.","url_abs":"https://arxiv.org/abs/2503.19470v2","url_pdf":"https://arxiv.org/pdf/2503.19470v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"research-learning-to-reason-with-search-for","repo_url":"https://github.com/agent-rl/research","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.19470","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.19470"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/agent-rl/research","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":2},"by_repo_kind":{"listed":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ad20cc71363546bd","entry":"compute_reinforce_plus_plus_outcome_advantage","repo":"agent-rl/research","repo_kind":"listed","path":"src/verl/trainer/ppo/core_algos.py","file_url":"https://github.com/agent-rl/research/blob/HEAD/src/verl/trainer/ppo/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad20cc71363546bd"}},{"code_sha256_prefix":"514a8131ed22efc6","entry":"masked_mean","repo":"agent-rl/research","repo_kind":"listed","path":"src/verl/trainer/ppo/core_algos.py","file_url":"https://github.com/agent-rl/research/blob/HEAD/src/verl/trainer/ppo/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"514a8131ed22efc6"}},{"code_sha256_prefix":"252438cc046bfb61","entry":"masked_var","repo":"agent-rl/research","repo_kind":"listed","path":"src/verl/trainer/ppo/core_algos.py","file_url":"https://github.com/agent-rl/research/blob/HEAD/src/verl/trainer/ppo/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"252438cc046bfb61"}},{"code_sha256_prefix":"4198da956b83f8be","entry":"masked_whiten","repo":"agent-rl/research","repo_kind":"listed","path":"src/verl/trainer/ppo/core_algos.py","file_url":"https://github.com/agent-rl/research/blob/HEAD/src/verl/trainer/ppo/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4198da956b83f8be"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}