{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stream-of-search-sos-learning-to-search-in","title":"Stream of Search (SoS): Learning to Search in Language","arxiv_id":"2404.03683","date":"2024-04-01","proceeding":null,"authors":["Kanishk Gandhi","Denise Lee","Gabriel Grand","Muxin Liu","Winson Cheng","Archit Sharma","Noah D. Goodman"],"abstract":"Language models are rarely shown fruitful mistakes while training. They then struggle to look beyond the next token, suffering from a snowballing of errors and struggling to predict the consequence of their actions several steps ahead. In this paper, we show how language models can be taught to search by representing the process of search in language, as a flattened string -- a stream of search (SoS). We propose a unified language for search that captures an array of different symbolic search strategies. We demonstrate our approach using the simple yet difficult game of Countdown, where the goal is to combine input numbers with arithmetic operations to reach a target number. We pretrain a transformer-based language model from scratch on a dataset of streams of search generated by heuristic solvers. We find that SoS pretraining increases search accuracy by 25% over models trained to predict only the optimal search trajectory. We further finetune this model with two policy improvement methods: Advantage-Induced Policy Alignment (APA) and Self-Taught Reasoner (STaR). The finetuned SoS models solve 36% of previously unsolved problems, including problems that cannot be solved by any of the heuristic solvers. Our results indicate that language models can learn to solve problems via search, self-improve to flexibly use different search strategies, and potentially discover new ones.","url_abs":"https://arxiv.org/abs/2404.03683v1","url_pdf":"https://arxiv.org/pdf/2404.03683v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stream-of-search-sos-learning-to-search-in","repo_url":"https://github.com/kanishkg/stream-of-search","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.03683","atlas_url":"https://app.syntology.ai/?focus=2404.03683","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.03683"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kanishkg/stream-of-search","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"088b5a4931c549e9","entry":"combine_nums","repo":"kanishkg/stream-of-search","repo_kind":"official","path":"src/countdown_utils.py","file_url":"https://github.com/kanishkg/stream-of-search/blob/HEAD/src/countdown_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"088b5a4931c549e9"}},{"code_sha256_prefix":"3b9eb49d3b384506","entry":"mult_heuristic","repo":"kanishkg/stream-of-search","repo_kind":"official","path":"src/countdown_utils.py","file_url":"https://github.com/kanishkg/stream-of-search/blob/HEAD/src/countdown_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b9eb49d3b384506"}},{"code_sha256_prefix":"f2e657b7b2f1263e","entry":"std_to_95_ci","repo":"kanishkg/stream-of-search","repo_kind":"official","path":"analysis/characterize_failures.py","file_url":"https://github.com/kanishkg/stream-of-search/blob/HEAD/analysis/characterize_failures.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f2e657b7b2f1263e"}},{"code_sha256_prefix":"221eded453668b01","entry":"sum_heuristic","repo":"kanishkg/stream-of-search","repo_kind":"official","path":"src/countdown_utils.py","file_url":"https://github.com/kanishkg/stream-of-search/blob/HEAD/src/countdown_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"221eded453668b01"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}