{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-program-with-natural-language","title":"Learning to Plan with Natural Language","arxiv_id":"2304.10464","date":"2023-04-20","proceeding":null,"authors":["Yiduo Guo","Yaobo Liang","Chenfei Wu","Wenshan Wu","Dongyan Zhao","Nan Duan"],"abstract":"Large Language Models (LLMs) have shown remarkable performance in various basic natural language tasks. For completing the complex task, we still need a plan for the task to guide LLMs to generate the specific solutions step by step. LLMs can directly generate task plans, but these plans may still contain factual errors or are incomplete. A high-quality task plan contains correct step-by-step solutions for solving all situations and behavioral instructions for avoiding mistakes. To obtain it, we propose the Learning to Plan method, which involves two phases: (1) In the first learning task plan phase, it iteratively updates the task plan with new step-by-step solutions and behavioral instructions, which are obtained by prompting LLMs to derive from training error feedback. (2) In the subsequent test phase, the LLM uses the learned task plan to guide the inference of LLM on the test set. We demonstrate the effectiveness of our method on the five different reasoning type tasks (8 datasets). Further, our analysis experiment shows that the task plan learned by one LLM can directly guide another LLM to improve its performance, which reveals a new transfer learning paradigm. We release the code at \\url{https://github.com/Eureka6174/LearnNLPlan}","url_abs":"https://arxiv.org/abs/2304.10464v4","url_pdf":"https://arxiv.org/pdf/2304.10464v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-program-with-natural-language","repo_url":"https://github.com/eureka6174/learnnlplan","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.10464","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.10464"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eureka6174/learnnlplan","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"f9b18a1da31d00fa","entry":"compress_prompt","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"prompts.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"f9b18a1da31d00fa"}},{"code_sha256_prefix":"d372acee011cefe5","entry":"few_shot_prompt","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"prompts.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"d372acee011cefe5"}},{"code_sha256_prefix":"426f6d200b9bf39a","entry":"generation_solution_prompt","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"prompts.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/prompts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"426f6d200b9bf39a"}},{"code_sha256_prefix":"0ecea1e9506dcc31","entry":"get_validation_set","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"utils.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"0ecea1e9506dcc31"}},{"code_sha256_prefix":"8c59349d9b4a534e","entry":"get_wrong_triples","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"utils.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"8c59349d9b4a534e"}},{"code_sha256_prefix":"4fc253f30bad52c9","entry":"query_azure_openai_chatgpt_chat","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"openai_public.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/openai_public.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"4fc253f30bad52c9"}},{"code_sha256_prefix":"382e7455732eb845","entry":"query_azure_openai_chatgpt_complete","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"openai_public.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/openai_public.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"382e7455732eb845"}},{"code_sha256_prefix":"0c026fdc99fb119e","entry":"shuffle_datapoints","repo":"eureka6174/learnnlplan","repo_kind":"official","path":"utils.py","file_url":"https://github.com/eureka6174/learnnlplan/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"0c026fdc99fb119e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}