{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/memory-augmented-policy-optimization-for","title":"Memory Augmented Policy Optimization for Program Synthesis and Semantic Parsing","arxiv_id":"1807.02322","date":"2018-07-06","proceeding":"NeurIPS 2018 12","authors":["Chen Liang","Mohammad Norouzi","Jonathan Berant","Quoc Le","Ni Lao"],"abstract":"We present Memory Augmented Policy Optimization (MAPO), a simple and novel\nway to leverage a memory buffer of promising trajectories to reduce the\nvariance of policy gradient estimate. MAPO is applicable to deterministic\nenvironments with discrete actions, such as structured prediction and\ncombinatorial optimization tasks. We express the expected return objective as a\nweighted sum of two terms: an expectation over the high-reward trajectories\ninside the memory buffer, and a separate expectation over trajectories outside\nthe buffer. To make an efficient algorithm of MAPO, we propose: (1) memory\nweight clipping to accelerate and stabilize training; (2) systematic\nexploration to discover high-reward trajectories; (3) distributed sampling from\ninside and outside of the memory buffer to scale up training. MAPO improves the\nsample efficiency and robustness of policy gradient, especially on tasks with\nsparse rewards. We evaluate MAPO on weakly supervised program synthesis from\nnatural language (semantic parsing). On the WikiTableQuestions benchmark, we\nimprove the state-of-the-art by 2.6%, achieving an accuracy of 46.3%. On the\nWikiSQL benchmark, MAPO achieves an accuracy of 74.9% with only weak\nsupervision, outperforming several strong baselines with full supervision. Our\nsource code is available at\nhttps://github.com/crazydonkey200/neural-symbolic-machines","url_abs":"http://arxiv.org/abs/1807.02322v5","url_pdf":"http://arxiv.org/pdf/1807.02322v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"memory-augmented-policy-optimization-for","repo_url":"https://github.com/crazydonkey200/neural-symbolic-machines","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"memory-augmented-policy-optimization-for","repo_url":"https://github.com/niansong1996/wassp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"memory-augmented-policy-optimization-for","repo_url":"https://github.com/pcyin/pytorch_neural_symbolic_machines","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"memory-augmented-policy-optimization-for","repo_url":"https://github.com/theSparta/neural-symbolic-machines","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"program-synthesis","task_name":"Program Synthesis"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"},{"task_slug":"structured-prediction","task_name":"Structured Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1807.02322","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1807.02322"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/crazydonkey200/neural-symbolic-machines","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/theSparta/neural-symbolic-machines","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pcyin/pytorch_neural_symbolic_machines","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/niansong1996/wassp","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":15},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1},"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c3104e5e2d8a40e4","entry":"attn_sum_bahdanau","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/tf_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/tf_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3104e5e2d8a40e4"}},{"code_sha256_prefix":"f1f2fe840e5c928e","entry":"attn_sum_dot","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/tf_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/tf_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f1f2fe840e5c928e"}},{"code_sha256_prefix":"9073ba6a227f3a75","entry":"bleu_score","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/nlp_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/nlp_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9073ba6a227f3a75"}},{"code_sha256_prefix":"88efa45be6ac471a","entry":"create_features","repo":"theSparta/neural-symbolic-machines","repo_kind":"listed","path":"nsm/env_factory.py","file_url":"https://github.com/theSparta/neural-symbolic-machines/blob/HEAD/nsm/env_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"88efa45be6ac471a"}},{"code_sha256_prefix":"ba6719f38bfb57e9","entry":"create_lm_inputs_labels","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/data_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba6719f38bfb57e9"}},{"code_sha256_prefix":"42891135110a96ff","entry":"create_multilayer_fnn","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/graph_factory.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/graph_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42891135110a96ff"}},{"code_sha256_prefix":"451d8aadbc0755dd","entry":"create_seq2seq_inputs","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/data_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"451d8aadbc0755dd"}},{"code_sha256_prefix":"ee764e83c41c8784","entry":"create_seq_mse_loss","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/graph_factory.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/graph_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee764e83c41c8784"}},{"code_sha256_prefix":"54fb9ba175e6c08b","entry":"edit_distance","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/nlp_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/nlp_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"54fb9ba175e6c08b"}},{"code_sha256_prefix":"fce4ea795a572f61","entry":"show_samples","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"table/experiment.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/table/experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fce4ea795a572f61"}},{"code_sha256_prefix":"cc1113db8b905bd0","entry":"tensormul","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/tf_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/tf_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cc1113db8b905bd0"}},{"code_sha256_prefix":"4fb20ccedc18c760","entry":"tokenize","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/nlp_utils.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/nlp_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4fb20ccedc18c760"}},{"code_sha256_prefix":"5677f59fb39ed6c9","entry":"unpack_program","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"table/experiment.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/table/experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5677f59fb39ed6c9"}},{"code_sha256_prefix":"9817d521556a77fa","entry":"value_to_index","repo":"theSparta/neural-symbolic-machines","repo_kind":"listed","path":"nsm/env_factory.py","file_url":"https://github.com/theSparta/neural-symbolic-machines/blob/HEAD/nsm/env_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9817d521556a77fa"}},{"code_sha256_prefix":"d34fbdeb6f1be262","entry":"with_graph_variable_scope","repo":"crazydonkey200/neural-symbolic-machines","repo_kind":"official","path":"nsm/graph_factory.py","file_url":"https://github.com/crazydonkey200/neural-symbolic-machines/blob/HEAD/nsm/graph_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d34fbdeb6f1be262"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}