{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/flow-of-reasoning-efficient-training-of-llm","title":"Flow of Reasoning:Training LLMs for Divergent Problem Solving with Minimal Examples","arxiv_id":"2406.05673","date":"2024-06-09","proceeding":null,"authors":["Fangxu Yu","Lai Jiang","Haoqiang Kang","Shibo Hao","Lianhui Qin"],"abstract":"The ability to generate diverse solutions to a given problem is a hallmark of human creativity. This divergent reasoning is also crucial for machines, enhancing their robustness and enabling them to assist humans in many applications such as scientific discovery. However, existing approaches to multi-step reasoning with large language models (LLMs) have mostly focused only on reasoning accuracy, without further discovering more diverse valid solutions. For example, supervised fine-tuning can improve LLM reasoning quality, but requires extensive supervised data to capture the full range of possible solutions. Reinforcement learning aims to find limited highest-reward solutions while neglecting the solution diversity. To fill this gap, we propose Flow of Reasoning (FoR), an efficient diversity-seeking LLM finetuning method aimed at improving reasoning quality and diversity with minimal data. FoR formulates multi-step LLM reasoning as a Markovian flow on a DAG-structured reasoning graph. This formulation allows us to incorporate and adapt principled GFlowNet approaches, for finetuning LLMs to sample diverse reasoning paths with probabilities proportional to the (unnormalized) reward of target problems. Extensive experiments show that, with limited training examples (e.g., 15 examples), FoR enables the discovery of diverse, creative, high-quality solutions, greatly outperforming a wide range of existing inference and training methods across five challenging puzzle-solving tasks, including BlocksWorld (embodied reasoning), Game24 (math puzzle solving), Rubik's Cube (spatial reasoning), 1D-ARC (abstraction reasoning), and PrOntoQA (logical reasoning). Code is available at https://github.com/Yu-Fangxu/FoR.","url_abs":"https://arxiv.org/abs/2406.05673v3","url_pdf":"https://arxiv.org/pdf/2406.05673v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"flow-of-reasoning-efficient-training-of-llm","repo_url":"https://github.com/yu-fangxu/for","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"arc","task_name":"ARC"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"logical-reasoning","task_name":"Logical Reasoning"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"rubik-s-cube","task_name":"Rubik's Cube"},{"task_slug":"spatial-reasoning","task_name":"Spatial Reasoning"},{"task_slug":"scientific-discovery","task_name":"scientific discovery"}],"methods":[{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"ppo","method_name":"PPO"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.05673","atlas_url":"https://app.syntology.ai/?focus=2406.05673","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.05673"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Yu-Fangxu/FoR","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yu-fangxu/for","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":11,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"afcd0eb6a3a4c39a","entry":"add_time","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"BlocksWorld/lightning_module_selection.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/BlocksWorld/lightning_module_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afcd0eb6a3a4c39a"}},{"code_sha256_prefix":"815d8e8d214865ea","entry":"color1_2_1","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Models.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"815d8e8d214865ea"}},{"code_sha256_prefix":"7226941042e90d9b","entry":"convert_tensors_to_numbers","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/lightning_module_selection.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/lightning_module_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7226941042e90d9b"}},{"code_sha256_prefix":"e4b13d74df59c1aa","entry":"detectFrontiers","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Task.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Task.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4b13d74df59c1aa"}},{"code_sha256_prefix":"2d983246aa9b1200","entry":"extract_last_state","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"prontoqa/lightning_module.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/prontoqa/lightning_module.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d983246aa9b1200"}},{"code_sha256_prefix":"8398b51b37d44019","entry":"extract_prompt","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"GSM8K/lightning_module_selection.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/GSM8K/lightning_module_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8398b51b37d44019"}},{"code_sha256_prefix":"012c0ca7e3ccba0e","entry":"get_class","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Utils.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"012c0ca7e3ccba0e"}},{"code_sha256_prefix":"327e54ca69883f06","entry":"is_equal","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"GSM8K/lightning_module_selection.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/GSM8K/lightning_module_selection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"327e54ca69883f06"}},{"code_sha256_prefix":"8f6dea1ad179b459","entry":"load_json_data","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/lightning_data.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/lightning_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f6dea1ad179b459"}},{"code_sha256_prefix":"df242a324791e4b9","entry":"pixelCorrespondence","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Models.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df242a324791e4b9"}},{"code_sha256_prefix":"085c428237ab0869","entry":"transform_to_function","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Utils.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"085c428237ab0869"}},{"code_sha256_prefix":"d22f365bc478b1f9","entry":"detectShapes","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Task.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Task.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d22f365bc478b1f9"}},{"code_sha256_prefix":"958485939c0541e6","entry":"detectShapesByColor","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Task.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Task.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"958485939c0541e6"}},{"code_sha256_prefix":"daabe65145c7218d","entry":"extract_last_state","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"Game24/lightning_module.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/Game24/lightning_module.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"daabe65145c7218d"}},{"code_sha256_prefix":"115cf95e952e04f2","entry":"get_tokenizer","repo":"Yu-Fangxu/FoR","repo_kind":"official","path":"1D-ARC/Utils.py","file_url":"https://github.com/Yu-Fangxu/FoR/blob/HEAD/1D-ARC/Utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"115cf95e952e04f2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}