{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/guided-stream-of-search-learning-to-better","title":"Guided Stream of Search: Learning to Better Search with Language Models via Optimal Path Guidance","arxiv_id":"2410.02992","date":"2024-10-03","proceeding":null,"authors":["Seungyong Moon","Bumsoo Park","Hyun Oh Song"],"abstract":"While language models have demonstrated impressive capabilities across a range of tasks, they still struggle with tasks that require complex planning and reasoning. Recent studies have proposed training language models on search processes rather than optimal solutions, resulting in better generalization performance even though search processes are noisy and even suboptimal. However, these studies overlook the value of optimal solutions, which can serve as step-by-step landmarks to guide more effective search. In this work, we explore how to leverage optimal solutions to enhance the search and planning abilities of language models. To this end, we propose guided stream of search (GSoS), which seamlessly incorporates optimal solutions into the self-generation process in a progressive manner, producing high-quality search trajectories. These trajectories are then distilled into the pre-trained model via supervised fine-tuning. Our approach significantly enhances the search and planning abilities of language models on Countdown, a simple yet challenging mathematical reasoning task. Notably, combining our method with RL fine-tuning yields further improvements, whereas previous supervised fine-tuning methods do not benefit from RL. Furthermore, our approach exhibits greater effectiveness than leveraging optimal solutions in the form of subgoal rewards.","url_abs":"https://arxiv.org/abs/2410.02992v1","url_pdf":"https://arxiv.org/pdf/2410.02992v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"guided-stream-of-search-learning-to-better","repo_url":"https://github.com/symoon11/guided-stream-of-search","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2410.02992","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.02992"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/snu-mllab/guided-rest","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/symoon11/guided-stream-of-search","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":17,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":10,"ran":9,"repositories":1},"found_in_text":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"088b5a4931c549e9","entry":"combine_nums","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/countdown_utils.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/countdown_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"088b5a4931c549e9"}},{"code_sha256_prefix":"f220595adfc5e83c","entry":"compute_ce_dpo_loss_rm","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/prime/prime_core_algos.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/prime/prime_core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f220595adfc5e83c"}},{"code_sha256_prefix":"4d91e4383716e42c","entry":"compute_detach_dpo_loss_rm","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/prime/prime_core_algos.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/prime/prime_core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4d91e4383716e42c"}},{"code_sha256_prefix":"f19825983a523003","entry":"compute_online_dpo_loss","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/spin/core_algos.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/spin/core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f19825983a523003"}},{"code_sha256_prefix":"41b6e81dc3ecba79","entry":"compute_onlinedpo_pref","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/spin/core_algos.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/spin/core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41b6e81dc3ecba79"}},{"code_sha256_prefix":"ce288b4b6be413e0","entry":"generate","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"tril/sample.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/tril/sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce288b4b6be413e0"}},{"code_sha256_prefix":"8896f4e2bafe8abc","entry":"generate_config_from_args","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"verl/model_merger/base_model_merger.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/verl/model_merger/base_model_merger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8896f4e2bafe8abc"}},{"code_sha256_prefix":"b2b11fca3d7fe4d1","entry":"get_child_nodes","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/countdown/core_algos.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/countdown/core_algos.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b2b11fca3d7fe4d1"}},{"code_sha256_prefix":"94209884dcb2308e","entry":"get_dynamic_pipeline_shards","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"verl/model_merger/megatron_model_merger.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/verl/model_merger/megatron_model_merger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"94209884dcb2308e"}},{"code_sha256_prefix":"757d109d90121142","entry":"get_kl_controller","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/spin/core_algos.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/spin/core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"757d109d90121142"}},{"code_sha256_prefix":"6ee6c56846603d81","entry":"get_prompts","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/sample.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ee6c56846603d81"}},{"code_sha256_prefix":"49715be63a40e887","entry":"get_search_paths","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/sample.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49715be63a40e887"}},{"code_sha256_prefix":"4893fcecbcbd648b","entry":"get_subgoal_nodes","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/augment.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/augment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4893fcecbcbd648b"}},{"code_sha256_prefix":"8551c0600f5ecd28","entry":"get_subgoals_list","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/augment.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/augment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8551c0600f5ecd28"}},{"code_sha256_prefix":"3b9eb49d3b384506","entry":"mult_heuristic","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/countdown_utils.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/countdown_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b9eb49d3b384506"}},{"code_sha256_prefix":"c6e7682bbaa501cb","entry":"parse_operation","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/augment.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/augment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6e7682bbaa501cb"}},{"code_sha256_prefix":"8a84f8cc921357e2","entry":"reward_func","repo":"snu-mllab/guided-rest","repo_kind":"found_in_text","path":"recipe/r1/reward_score.py","file_url":"https://github.com/snu-mllab/guided-rest/blob/HEAD/recipe/r1/reward_score.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8a84f8cc921357e2"}},{"code_sha256_prefix":"221eded453668b01","entry":"sum_heuristic","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/countdown_utils.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/countdown_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"221eded453668b01"}},{"code_sha256_prefix":"9b5682ed4efd3768","entry":"generate","repo":"symoon11/guided-stream-of-search","repo_kind":"official","path":"stream-of-search/src/sample.py","file_url":"https://github.com/symoon11/guided-stream-of-search/blob/HEAD/stream-of-search/src/sample.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b5682ed4efd3768"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}