{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/copiloting-the-copilots-fusing-large-language","title":"Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair","arxiv_id":"2309.00608","date":"2023-09-01","proceeding":null,"authors":["Yuxiang Wei","Chunqiu Steven Xia","Lingming Zhang"],"abstract":"During Automated Program Repair (APR), it can be challenging to synthesize correct patches for real-world systems in general-purpose programming languages. Recent Large Language Models (LLMs) have been shown to be helpful \"copilots\" in assisting developers with various coding tasks, and have also been directly applied for patch synthesis. However, most LLMs treat programs as sequences of tokens, meaning that they are ignorant of the underlying semantics constraints of the target programming language. This results in plenty of statically invalid generated patches, impeding the practicality of the technique. Therefore, we propose Repilot, a general code generation framework to further copilot the AI \"copilots\" (i.e., LLMs) by synthesizing more valid patches during the repair process. Our key insight is that many LLMs produce outputs autoregressively (i.e., token by token), resembling human writing programs, which can be significantly boosted and guided through a Completion Engine. Repilot synergistically synthesizes a candidate patch through the interaction between an LLM and a Completion Engine, which 1) prunes away infeasible tokens suggested by the LLM and 2) proactively completes the token based on the suggestions provided by the Completion Engine. Our evaluation on a subset of the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot outperforms state-of-the-art techniques by fixing 27% and 47% more bugs, respectively. Moreover, Repilot produces more valid and correct patches than the base LLM with the same budget. While we focus on leveraging Repilot for APR in this work, the overall approach is also generalizable to other code generation tasks.","url_abs":"https://arxiv.org/abs/2309.00608v3","url_pdf":"https://arxiv.org/pdf/2309.00608v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"copiloting-the-copilots-fusing-large-language","repo_url":"https://github.com/ise-uiuc/Repilot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"program-repair","task_name":"Program Repair"},{"task_slug":null,"task_name":"valid"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"repair","method_name":"Repair"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.00608","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.00608"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ise-uiuc/Repilot","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":7},"by_repo_kind":{"official":{"samples":7,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a89140b43c639226","entry":"char_may_trigger_completion","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/generation.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/generation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a89140b43c639226"}},{"code_sha256_prefix":"164175bb44f84216","entry":"match_calling_function","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/template.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/template.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"164175bb44f84216"}},{"code_sha256_prefix":"fe5517e31bea5992","entry":"match_conditional_expression","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/template.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/template.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5517e31bea5992"}},{"code_sha256_prefix":"a7239965ae6c0780","entry":"match_function_api_call","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/template.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/template.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a7239965ae6c0780"}},{"code_sha256_prefix":"ff2facb100494948","entry":"take_while","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/utils.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff2facb100494948"}},{"code_sha256_prefix":"529ab9928631fcbf","entry":"take_while_two","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/utils.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"529ab9928631fcbf"}},{"code_sha256_prefix":"0d279f41ed1fdeef","entry":"transpose","repo":"ise-uiuc/Repilot","repo_kind":"official","path":"src/repilot/evaluation.py","file_url":"https://github.com/ise-uiuc/Repilot/blob/HEAD/src/repilot/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0d279f41ed1fdeef"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}