{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/progressive-hint-prompting-improves-reasoning","title":"Progressive-Hint Prompting Improves Reasoning in Large Language Models","arxiv_id":"2304.09797","date":"2023-04-19","proceeding":null,"authors":["Chuanyang Zheng","Zhengying Liu","Enze Xie","Zhenguo Li","Yu Li"],"abstract":"The performance of Large Language Models (LLMs) in reasoning tasks depends heavily on prompt design, with Chain-of-Thought (CoT) and self-consistency being critical methods that enhance this ability. However, these methods do not fully exploit the answers generated by the LLM to guide subsequent responses. This paper proposes a new prompting method, named Progressive-Hint Prompting (PHP), that enables automatic multiple interactions between users and LLMs by using previously generated answers as hints to progressively guide toward the correct answers. PHP is orthogonal to CoT and self-consistency, making it easy to combine with state-of-the-art techniques to further improve performance. We conducted extensive and comprehensive experiments on seven benchmarks. The results show that PHP significantly improves accuracy while remaining highly efficient. For instance, with text-davinci-003, we observed a 4.2% improvement on GSM8K with greedy decoding compared to Complex CoT, and a 46.17% reduction in sample paths with self-consistency. With GPT-4 and PHP, we achieve state-of-the-art performances on SVAMP (89.1% -> 91.9%), GSM8K (92% -> 95.5%), AQuA (76.4% -> 79.9%) and MATH (50.3% -> 53.9%).","url_abs":"https://arxiv.org/abs/2304.09797v6","url_pdf":"https://arxiv.org/pdf/2304.09797v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"progressive-hint-prompting-improves-reasoning","repo_url":"https://github.com/chuanyang-Zheng/Progressive-Hint","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"arithmetic-reasoning","task_name":"Arithmetic Reasoning"},{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"},{"task_slug":"math-word-problem-solving","task_name":"Math Word Problem Solving"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/math-word-problem-solving-on-math","task":"Math Word Problem Solving","dataset":"MATH","model":"PHP (GPT-4 model)","rank_in_archive_order":39,"of":135,"metrics":{"Accuracy":"53.9"},"uses_additional_data":false},{"leaderboard":"/sota/math-word-problem-solving-on-svamp","task":"Math Word Problem Solving","dataset":"SVAMP","model":"GPT-4 (PHP)","rank_in_archive_order":4,"of":26,"metrics":{"Execution Accuracy":"91.9"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2304.09797","atlas_url":"https://app.syntology.ai/?focus=2304.09797","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.09797"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chuanyang-Zheng/Progressive-Hint","reach":null}],"summary":{"ran_draft_wrong":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"89a79a5b4ecab38b","entry":"create_repeat_list","repo":"chuanyang-Zheng/Progressive-Hint","repo_kind":"official","path":"main_clean.py","file_url":"https://github.com/chuanyang-Zheng/Progressive-Hint/blob/HEAD/main_clean.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89a79a5b4ecab38b"}},{"code_sha256_prefix":"a38fabc415bd4850","entry":"create_response","repo":"chuanyang-Zheng/Progressive-Hint","repo_kind":"official","path":"main_clean.py","file_url":"https://github.com/chuanyang-Zheng/Progressive-Hint/blob/HEAD/main_clean.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a38fabc415bd4850"}},{"code_sha256_prefix":"4a6ddc88258d736d","entry":"create_response_chat","repo":"chuanyang-Zheng/Progressive-Hint","repo_kind":"official","path":"main_clean.py","file_url":"https://github.com/chuanyang-Zheng/Progressive-Hint/blob/HEAD/main_clean.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a6ddc88258d736d"}},{"code_sha256_prefix":"6ef9939f98a8e424","entry":"get_answer_from_gpt_sample","repo":"chuanyang-Zheng/Progressive-Hint","repo_kind":"official","path":"main_clean.py","file_url":"https://github.com/chuanyang-Zheng/Progressive-Hint/blob/HEAD/main_clean.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ef9939f98a8e424"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}