Browse State-of-the-Art › Program Repair
Program Repair
52 papers with code · 4 benchmarks · 9 datasets archive 2025-07-28
Task of teaching ML models to modify an existing program to fix a bug in a given code.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
4 leaderboard tables shown for this task, 4 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
9 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
6 subtasks in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 132 · with a code link: 52 · where Syntology ran a sample: 18 (16 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (18 of 132 tagged: 16 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 132, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
8 Apr 2024 5 repositories listed Syntology official (archive's flag): 1 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 1 pointer-only (licence)Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding.
-
1 Jul 2024 4 repositories listed Syntology official (archive's flag): 3 ran · 15 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 1 where Syntology's instrument failed) · 9 unverified (of 24 harvested samples) · 1 pointer-only (licence)However, the complexity of these agent-based approaches, together with the limited abilities of current LLMs, raises the following question: Do we really have to employ complex autonomous software agents?
-
6 Mar 2023 4 repositories listed Syntology official (archive's flag): 4 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)Recently, pre-trained large language models (LLMs) have shown impressive abilities in generating codes from natural language descriptions, repairing buggy codes, translating codes between languages, and retrieving…
-
17 Jun 2022 3 repositories listedDue to the vast number of students enrolled in Massive Open Online Courses (MOOCs), there has been an increasing number of automated program repair techniques focused on introductory programming assignments (IPAs).
-
27 Sep 2024 2 repositories listedThe key characteristics of RepairBench are: 1) it is execution-based: all patches are compiled and executed against a test suite, 2) it assesses frontier models in a frequent and standardized way.
-
9 Sep 2024 2 repositories listedLarge Language Models (LLMs) have revolutionized software engineering (SE), showcasing remarkable proficiency in various coding tasks.
-
In particular, we demonstrate that Self-Debugging can teach the large language model to perform rubber duck debugging; i. e., without any human feedback on the code correctness or error messages, the model is able to…
-
31 Mar 2021 2 repositories listedDespite the immense popularity of the Automated Program Repair (APR) field, the question of patch validation is still open.
-
10 Mar 2021 2 repositories listedExperiments on code summarization in the English language, code generation, and code translation in seven programming languages show that PLBART outperforms or rivals state-of-the-art models.
-
20 May 2020 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)Second, we present a self-supervised learning paradigm for program repair that leverages unlabeled programs available online to create a large amount of extra program repair examples, which we use to pre-train our…
-
21 Dec 2019 2 repositories listedWe fine-tune CuBERT on our benchmark tasks, and compare the resulting models to different variants of Word2Vec token embeddings, BiLSTM and Transformer models, as well as published state-of-the-art models, showing that…
-
3 Apr 2019 2 repositories listed Syntology 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)We show that it is beneficial to train a model that jointly and directly localizes and repairs variable-misuse bugs.
-
24 Dec 2018 2 repositories listedThis paper presents a novel end-to-end approach to program repair based on sequence-to-sequence learning.
-
28 Mar 2025 1 repository listedCoSIL reduces the search space through module call graphs, iteratively searches the function call graph to obtain relevant contexts, and uses context pruning to control the search direction and manage contexts…
-
9 Mar 2025 1 repository listedAutomated Program Repair (APR) is a task to automatically generate patches for the buggy code.
-
30 Jan 2025 1 repository listedThe irruption of DeepSeek-R1 constitutes a turning point for the AI industry in general and the LLMs in particular.
-
5 Dec 2024 1 repository listedWe propose DEVLoRe to use issue content (description and message) and stack error traces to localize buggy methods, then rely on debug information in buggy methods and issue content and stack error to localize buggy…
-
21 Nov 2024 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 9 harvested samples)LPW also sets new state-of-the-art Pass@1 accuracy, achieving 98.
-
2 Oct 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples)While large language models have made significant strides in code generation, the pass rate of the generated code is bottlenecked on subtle errors, often requiring human intervention to pass tests, especially for…
-
16 Sep 2024 1 repository listedLLMs have long demonstrated remarkable effectiveness in automatic program repair (APR), with OpenAI's ChatGPT being one of the most widely used models in this domain.
-
21 Aug 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)The gap between the trepidation of program reliability and the expense of repairs underscores the indispensability of Automated Program Repair (APR).
-
3 Jun 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 0 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 3 unverified (of 15 harvested samples)This approach led to the development of SemCoder, a Code LLM with only 6.
-
8 May 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 0 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (of 13 harvested samples) · 13 pointer-only (licence)The emergence of large language models (LLMs) has sparked enormous interest due to their potential application across a range of educational tasks.
-
13 Apr 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples)In this paper, we investigate a new approach to adapt LLMs to program repair.
-
25 Mar 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Unlike existing deep learning-based approaches, which prompt a model with a fixed prompt or in a fixed feedback loop, our work treats the LLM as an agent capable of autonomously planning and executing actions to fix…
-
19 Feb 2024 1 repository listedIn this paper, we propose shifting the focus of robustness evaluation for Neural Program Repair (NPR) techniques toward naturally-occurring data transformations.
-
15 Jan 2024 1 repository listedWe investigate whether this correction capability of Large Language Models (LLMs) extends to Automatic Program Repair (APR).
-
25 Dec 2023 1 repository listed Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)This results in RepairLLaMA producing a highly effective `program repair adapter' for fixing bugs with AI.
-
13 Dec 2023 1 repository listedLarge Language Models (LLMs) have gained considerable traction within the Software Engineering (SE) community, impacting various SE tasks from code completion to test generation, from program repair to code…
-
8 Dec 2023 1 repository listedOur results indicate that overall repair success increases with the size of the local context (albeit not for all bug types) and confirm the common practice that roughly 50-60% of the input window should be used for…
-
18 Oct 2023 1 repository listed
-
1 Sep 2023 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)
-
24 Jul 2023 1 repository listed
-
29 May 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
3 Feb 2023 1 repository listed
-
3 Jan 2023 1 repository listed
-
28 Jun 2022 1 repository listed
-
18 Jul 2021 1 repository listed
-
15 Jun 2021 1 repository listed Syntology 12 ran (of which 1 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 1 violated, 8 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 14 harvested samples) · 1 pointer-only (licence)
-
11 Jun 2021 1 repository listed
-
11 Jun 2021 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)
-
26 Feb 2021 1 repository listed
-
23 Oct 2020 1 repository listed
-
24 Aug 2020 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 1 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples)
-
10 Aug 2020 1 repository listed
-
18 Jul 2020 1 repository listed
-
1 May 2020 1 repository listed
-
28 Dec 2019 1 repository listed
-
16 Dec 2019 1 repository listed
-
31 Jan 2018 1 repository listed
-
20 Nov 2017 1 repository listed
-
4 Feb 2017 1 repository listed
-
CORE: Benchmarking LLMs Code Reasoning Capabilities through Static Analysis Tasks3 Jul 2025 0 repositories listed
-
T³: Multi-level Tree-based Automatic Program Repair with Large Language Models26 Jun 2025 0 repositories listed
-
Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories23 Jun 2025 0 repositories listed
-
Dissecting the SWE-Bench Leaderboards: Profiling Submitters and Architectures of LLM- and Agent-Based Repair Systems20 Jun 2025 0 repositories listed
-
SemAgent: A Semantics Aware Program Repair Agent19 Jun 2025 0 repositories listed
-
A Multi-Dataset Evaluation of Models for Automated Vulnerability Repair5 Jun 2025 0 repositories listed
-
An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks27 May 2025 0 repositories listed
-
Gradient-Based Program Repair: Fixing Bugs in Continuous Program Spaces23 May 2025 0 repositories listed
-
Synthetic Code Surgery: Repairing Bugs and Vulnerabilities with LLMs and Synthetic Data12 May 2025 0 repositories listed
-
Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs7 May 2025 0 repositories listed
-
The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models5 May 2025 0 repositories listed
-
SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs20 Apr 2025 0 repositories listed
-
Using ML filters to help automated vulnerability repairs: when it helps and when it doesn't9 Apr 2025 0 repositories listed
-
Attention Pruning: Automated Fairness Repair of Language Models via Surrogate Simulated Annealing20 Mar 2025 0 repositories listed
-
Evaluating the Generalizability of LLMs in Automated Program Repair12 Mar 2025 0 repositories listed
-
Where's the Bug? Attention Probing for Scalable Fault Localization19 Feb 2025 0 repositories listed
-
LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks10 Feb 2025 0 repositories listed
-
Agentic Bug Reproduction for Effective Automated Program Repair at Google3 Feb 2025 0 repositories listed
-
Evaluating Agent-based Program Repair at Google13 Jan 2025 0 repositories listed
-
The Impact of Input Order Bias on Large Language Models for Software Fault Localization25 Dec 2024 0 repositories listed
-
Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization19 Dec 2024 0 repositories listed
-
A Comprehensive Survey of AI-Driven Advancements and Techniques in Automated Program Repair and Code Generation12 Nov 2024 0 repositories listed
-
MdEval: Massively Multilingual Code Debugging4 Nov 2024 0 repositories listed
-
Semantic-guided Search for Efficient Program Repair with Large Language Models22 Oct 2024 0 repositories listed
-
Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code13 Oct 2024 0 repositories listed
-
In-Context Code-Text Learning for Bimodal Software Engineering8 Oct 2024 0 repositories listed
-
Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench6 Oct 2024 0 repositories listed
-
Enhancing Automated Program Repair with Solution Design22 Aug 2024 0 repositories listed
-
MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair18 Aug 2024 0 repositories listed
-
SpecRover: Code Intent Extraction via LLMs5 Aug 2024 0 repositories listed
-
Automated C/C++ Program Repair for High-Level Synthesis via Large Language Models4 Jul 2024 0 repositories listed
-
NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repair24 Jun 2024 0 repositories listed
-
Automated Program Repair: Emerging trends pose and expose problems for benchmarks8 May 2024 0 repositories listed
-
Automatic Programming: Large Language Models and Beyond3 May 2024 0 repositories listed
-
NExT: Teaching Large Language Models to Reason about Code Execution23 Apr 2024 0 repositories listed
-
Peer-aided Repairer: Empowering Large Language Models to Repair Advanced Student Assignments2 Apr 2024 0 repositories listed
-
To Err is Machine: Vulnerability Detection Challenges LLM Reasoning25 Mar 2024 0 repositories listed
-
A Study of Vulnerability Repair in JavaScript Programs with Large Language Models19 Mar 2024 0 repositories listed
-
DeepCode AI Fix: Fixing Security Vulnerabilities with Large Language Models19 Feb 2024 0 repositories listed
-
Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning22 Nov 2023 0 repositories listed
-
ConDefects: A New Dataset to Address the Data Leakage Concern for LLM-based Fault Localization and Program Repair25 Oct 2023 0 repositories listed
-
Automated Bug Generation in the era of Large Language Models3 Oct 2023 0 repositories listed
-
Program Repair with Minimal Edits Using CodeT526 Sep 2023 0 repositories listed
-
Frustrated with Code Quality Issues? LLMs can Help!22 Sep 2023 0 repositories listed
-
RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair12 Sep 2023 0 repositories listed
-
An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code5 Jul 2023 0 repositories listed
-
Better patching using LLM prompting, via Self-Consistency31 May 2023 0 repositories listed
-
Is ChatGPT the Ultimate Programming Assistant -- How far is it?24 Apr 2023 0 repositories listed
Syntology lines on 19 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections