{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/synthesize-execute-and-debug-learning-to","title":"Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis","arxiv_id":"2007.08095","date":"2020-07-16","proceeding":"NeurIPS 2020 12","authors":["Kavi Gupta","Peter Ebert Christensen","Xinyun Chen","Dawn Song"],"abstract":"The use of deep learning techniques has achieved significant progress for program synthesis from input-output examples. However, when the program semantics become more complex, it still remains a challenge to synthesize programs that are consistent with the specification. In this work, we propose SED, a neural program generation framework that incorporates synthesis, execution, and debugging stages. Instead of purely relying on the neural program synthesizer to generate the final program, SED first produces initial programs using the neural program synthesizer component, then utilizes a neural program debugger to iteratively repair the generated programs. The integration of the debugger component enables SED to modify the programs based on the execution results and specification, which resembles the coding process of human programmers. On Karel, a challenging input-output program synthesis benchmark, SED reduces the error rate of the neural program synthesizer itself by a considerable margin, and outperforms the standard beam search for decoding.","url_abs":"https://arxiv.org/abs/2007.08095v2","url_pdf":"https://arxiv.org/pdf/2007.08095v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"synthesize-execute-and-debug-learning-to","repo_url":"https://github.com/sunblaze-ucb/SED","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"program-synthesis","task_name":"Program Synthesis"}],"methods":[{"method_slug":"repair","method_name":"Repair"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.08095","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.08095"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunblaze-ucb/SED","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":19},"by_repo_kind":{"listed":{"samples":19,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"79c5f129bde8c733","entry":"by_max_acc","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/analysis/organize_results.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/analysis/organize_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79c5f129bde8c733"}},{"code_sha256_prefix":"5f53ac6d6dd92573","entry":"by_max_step","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/analysis/organize_results.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/analysis/organize_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f53ac6d6dd92573"}},{"code_sha256_prefix":"039979ec200d5319","entry":"code_stat","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/stats.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"039979ec200d5319"}},{"code_sha256_prefix":"fef08668e5cc872a","entry":"evaluate_code","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/executor.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/executor.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fef08668e5cc872a"}},{"code_sha256_prefix":"00ffb35e6b02915b","entry":"expand_by_beam","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/models/beam_search.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/models/beam_search.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"00ffb35e6b02915b"}},{"code_sha256_prefix":"881122c4654562fb","entry":"get_arg_parser","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/arguments.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/arguments.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"881122c4654562fb"}},{"code_sha256_prefix":"a09108fab4b8d1b4","entry":"get_attn_mask","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/models/base.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/models/base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a09108fab4b8d1b4"}},{"code_sha256_prefix":"7eeeb9e1399f3c5c","entry":"get_exact_match","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/analysis/load_results.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/analysis/load_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7eeeb9e1399f3c5c"}},{"code_sha256_prefix":"74ee36a2c213a620","entry":"get_rev_vocab","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/data.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74ee36a2c213a620"}},{"code_sha256_prefix":"5f7285b3d57507d8","entry":"hyp_score","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/models/beam_search.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/models/beam_search.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f7285b3d57507d8"}},{"code_sha256_prefix":"d5825e840b5f66b7","entry":"karel_output_ref_mutator_mode","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/dataset.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5825e840b5f66b7"}},{"code_sha256_prefix":"918b15bc296f5115","entry":"length_penalty","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/models/beam_search.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/models/beam_search.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"918b15bc296f5115"}},{"code_sha256_prefix":"b85637c0fc6f9373","entry":"lisp_code_stat","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/stats.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b85637c0fc6f9373"}},{"code_sha256_prefix":"3d825f33d60f08a3","entry":"load_vocab","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/data.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d825f33d60f08a3"}},{"code_sha256_prefix":"5957df89a5e9f359","entry":"relpath","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/dataset.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5957df89a5e9f359"}},{"code_sha256_prefix":"22048f44250fc46a","entry":"replace_pad_with_end","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/data.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22048f44250fc46a"}},{"code_sha256_prefix":"5c5394d7e87f8bab","entry":"separate_strategies","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/analysis/organize_results.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/analysis/organize_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c5394d7e87f8bab"}},{"code_sha256_prefix":"2e635b1a8221f10c","entry":"table_of_accuracies","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/analysis/load_results.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/analysis/load_results.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e635b1a8221f10c"}},{"code_sha256_prefix":"3c24d293db334783","entry":"test_cardinality","repo":"sunblaze-ucb/SED","repo_kind":"listed","path":"program_synthesis/datasets/stats.py","file_url":"https://github.com/sunblaze-ucb/SED/blob/HEAD/program_synthesis/datasets/stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c24d293db334783"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}