{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-prompt-learning-via-policy-gradient","title":"Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning","arxiv_id":"2209.14610","date":"2022-09-29","proceeding":null,"authors":["Pan Lu","Liang Qiu","Kai-Wei Chang","Ying Nian Wu","Song-Chun Zhu","Tanmay Rajpurohit","Peter Clark","Ashwin Kalyan"],"abstract":"Mathematical reasoning, a core ability of human intelligence, presents unique challenges for machines in abstract thinking and logical reasoning. Recent large pre-trained language models such as GPT-3 have achieved remarkable progress on mathematical reasoning tasks written in text form, such as math word problems (MWP). However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data. To fill the gap, we present Tabular Math Word Problems (TabMWP), a new dataset containing 38,431 open-domain grade-level problems that require mathematical reasoning on both textual and tabular data. Each question in TabMWP is aligned with a tabular context, which is presented as an image, semi-structured text, and a structured table. There are two types of questions: free-text and multi-choice, and each problem is annotated with gold solutions to reveal the multi-step reasoning process. We evaluate different pre-trained models on TabMWP, including the GPT-3 model in a few-shot setting. As earlier studies suggest, since few-shot GPT-3 relies on the selection of in-context examples, its performance is unstable and can degrade to near chance. The unstable issue is more severe when handling complex problems like TabMWP. To mitigate this, we further propose a novel approach, PromptPG, which utilizes policy gradient to learn to select in-context examples from a small amount of training data and then constructs the corresponding prompt for the test example. Experimental results show that our method outperforms the best baseline by 5.31% on the accuracy metric and reduces the prediction variance significantly compared to random selection, which verifies its effectiveness in selecting in-context examples.","url_abs":"https://arxiv.org/abs/2209.14610v3","url_pdf":"https://arxiv.org/pdf/2209.14610v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-prompt-learning-via-policy-gradient","repo_url":"https://github.com/lupantech/promptpg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"dynamic-prompt-learning-via-policy-gradient","repo_url":"https://github.com/opendilab/DI-engine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"logical-reasoning","task_name":"Logical Reasoning"},{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-3","method_name":"GPT-3"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"test","method_name":"Test"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.14610","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.14610"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lupantech/promptpg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opendilab/DI-engine","reach":null}],"summary":{"ran_draft_wrong":2,"ran_honours":1,"unverified":4},"by_repo_kind":{"listed":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"03b16328d15c5de2","entry":"extract_prediction","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03b16328d15c5de2"}},{"code_sha256_prefix":"580caf49d1cd4e8e","entry":"normalize_answer","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"580caf49d1cd4e8e"}},{"code_sha256_prefix":"cc627ffb4d816873","entry":"score_string_similarity","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc627ffb4d816873"}},{"code_sha256_prefix":"afad6bcff6e4c6ce","entry":"call_gpt3","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afad6bcff6e4c6ce"}},{"code_sha256_prefix":"63b8daf675dc9a6a","entry":"get_batch_reward_loss","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63b8daf675dc9a6a"}},{"code_sha256_prefix":"92646ee5d73f558a","entry":"get_gpt3_output","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92646ee5d73f558a"}},{"code_sha256_prefix":"38fcc5eb9e91cdb9","entry":"policy_gradient_train","repo":"lupantech/promptpg","repo_kind":"listed","path":"run_gpt3_rl/learn_policy.py","file_url":"https://github.com/lupantech/promptpg/blob/HEAD/run_gpt3_rl/learn_policy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38fcc5eb9e91cdb9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}