{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/from-code-to-correctness-closing-the-last","title":"From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging","arxiv_id":"2410.01215","date":"2024-10-02","proceeding":null,"authors":["Yuling Shi","Songsong Wang","Chengcheng Wan","Xiaodong Gu"],"abstract":"While large language models have made significant strides in code generation, the pass rate of the generated code is bottlenecked on subtle errors, often requiring human intervention to pass tests, especially for complex problems. Existing LLM-based debugging systems treat generated programs as monolithic units, failing to address bugs at multiple levels of granularity, from low-level syntax errors to high-level algorithmic flaws. In this paper, we introduce Multi-Granularity Debugger (MGDebugger), a hierarchical code debugger by isolating, identifying, and resolving bugs at various levels of granularity. MGDebugger decomposes problematic code into a hierarchical tree structure of subfunctions, with each level representing a particular granularity of error. During debugging, it analyzes each subfunction and iteratively resolves bugs in a bottom-up manner. To effectively test each subfunction, we propose an LLM-simulated Python executor, which traces code execution and tracks important variable states to pinpoint errors accurately. Extensive experiments demonstrate that MGDebugger outperforms existing debugging systems, achieving an 18.9% improvement in accuracy over seed generations in HumanEval and a 97.6% repair success rate in HumanEvalFix. Furthermore, MGDebugger effectively fixes bugs across different categories and difficulty levels, demonstrating its robustness and effectiveness.","url_abs":"https://arxiv.org/abs/2410.01215v2","url_pdf":"https://arxiv.org/pdf/2410.01215v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"from-code-to-correctness-closing-the-last","repo_url":"https://github.com/YerbaPage/MGDebugger","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"auto-debugging","task_name":"Auto Debugging"},{"task_slug":"bug-fixing","task_name":"Bug fixing"},{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"humaneval","task_name":"HumanEval"},{"task_slug":"program-repair","task_name":"Program Repair"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/code-generation-on-humaneval","task":"Code Generation","dataset":"HumanEval","model":"DeepSeek-R1 (MGDebugger)","rank_in_archive_order":1,"of":8,"metrics":{"Pass@1":"100"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-mbpp","task":"Code Generation","dataset":"MBPP","model":"MGDebugger (DeepSeek-V3-0324)","rank_in_archive_order":4,"of":99,"metrics":{"Accuracy":"92.4"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-mbpp","task":"Code Generation","dataset":"MBPP","model":"MGDebugger (CodeQwen1.5)","rank_in_archive_order":21,"of":99,"metrics":{"Accuracy":"80.8"},"uses_additional_data":false},{"leaderboard":"/sota/program-repair-on-humanevalpack","task":"Program Repair","dataset":"HumanEvalPack","model":"MGDebugger (DeepSeek-Coder-V2-Lite)","rank_in_archive_order":1,"of":1,"metrics":{"Pass@1":"97.6"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.01215","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.01215"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/YerbaPage/MGDebugger","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e037404ea645cd8f","entry":"create_dependency_graph","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e037404ea645cd8f"}},{"code_sha256_prefix":"bcdb4977bf68667c","entry":"function_with_timeout","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/timeout_utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/timeout_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcdb4977bf68667c"}},{"code_sha256_prefix":"c85ba1046609f5b4","entry":"get_parameter_names","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/testcase_utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/testcase_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c85ba1046609f5b4"}},{"code_sha256_prefix":"dbf7462bc9cefeaf","entry":"merge_changes_to_parents","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbf7462bc9cefeaf"}},{"code_sha256_prefix":"2c7d6590d386f8d9","entry":"parse_tests","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/testcase_utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/testcase_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c7d6590d386f8d9"}},{"code_sha256_prefix":"4880ae3be078f0c3","entry":"process_assert","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/testcase_utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/testcase_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4880ae3be078f0c3"}},{"code_sha256_prefix":"efd6500fc32bf1eb","entry":"set_memory_limit","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/timeout_utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/timeout_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efd6500fc32bf1eb"}},{"code_sha256_prefix":"a37aeb14211cb8a6","entry":"topological_sort","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a37aeb14211cb8a6"}},{"code_sha256_prefix":"9a0e376a4327ffd4","entry":"get_completion_with_retry","repo":"YerbaPage/MGDebugger","repo_kind":"official","path":"baseline/common.py","file_url":"https://github.com/YerbaPage/MGDebugger/blob/HEAD/baseline/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a0e376a4327ffd4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}