Bug fixing

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 31 papers with code (62 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• GPT-4 Technical Report
  15 Mar 2023 11 repositories listed Syntology ran 2 of 5 samples · 3 unverified · 1 pointer-only (licence)
  We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
  10 Oct 2023 8 repositories listed Syntology ran 0 of 5 samples · 5 unverified
  We find real-world software engineering to be a rich, sustainable, and challenging testbed for evaluating the next generation of language models.
• AutoCodeRover: Autonomous Program Improvement
  8 Apr 2024 5 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 1 pointer-only (licence)
  Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding.
• Neural Transfer Learning for Repairing Security Vulnerabilities in C Code
  16 Apr 2021 3 repositories listed
  To sum up, this paper shows that transfer learning works well for repairing security vulnerabilities in C compared to learning on a small dataset.
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
  6 May 2024 2 repositories listed
  We investigate how interface design affects the performance of language model agents.
• CoreCodeBench: A Configurable Multi-Scenario Repository-Level Benchmark
  4 Jul 2025 1 repository listed
  As Large Language Models (LLMs) demonstrate increasingly sophisticated code processing capabilities, evaluating their performance on engineering-level code remains challenging.
• SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Development
  22 May 2025 1 repository listed Syntology ran 5 of 5 samples · 0 unverified · 5 pointer-only (licence)
  We therefore introduce SWE-Dev, the first large-scale dataset (with 14, 000 training and 500 test samples) designed to evaluate and train autonomous coding systems on real-world feature development tasks.
• Less is More: Adaptive Program Repair with Bug Localization and Preference Learning
  9 Mar 2025 1 repository listed
  Automated Program Repair (APR) is a task to automatically generate patches for the buggy code.
• Repository-level Code Search with Neural Retrieval Methods
  10 Feb 2025 1 repository listed
  We define the task of repository-level code search as retrieving the set of files from the current state of a code repository that are most relevant to addressing a user's question or bug.
• GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation
  19 Jan 2025 1 repository listed
  Results show that our method reduces the energy consumption between 23-50 % on average for code generation tasks without significantly affecting accuracy.
• CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking
  1 Dec 2024 1 repository listed Syntology ran 0 of 14 samples · 14 unverified
  Effective code retrieval plays a crucial role in advancing code generation, bug fixing, and software maintenance, particularly as software systems increase in complexity.
• MetRex: A Benchmark for Verilog Code Metric Reasoning Using LLMs
  5 Nov 2024 1 repository listed Syntology ran 0 of 13 samples · 13 unverified · 13 pointer-only (licence)
  In this paper, we assess the ability of LLMs to reason about post-synthesis metrics of Verilog designs.
• From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging
  2 Oct 2024 1 repository listed Syntology ran 8 of 9 samples · 1 unverified
  While large language models have made significant strides in code generation, the pass rate of the generated code is bottlenecked on subtle errors, often requiring human intervention to pass tests, especially for…
• Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests
  21 Aug 2024 1 repository listed
  Automated unit test generators, particularly search-based software testing tools like EvoSuite, are capable of generating tests with high coverage.
• Patched RTC: evaluating LLMs for diverse software development tasks
  23 Jul 2024 1 repository listed
  This paper introduces Patched Round-Trip Correctness (Patched RTC), a novel evaluation technique for Large Language Models (LLMs) applied to diverse software development tasks, particularly focusing on "outer loop"…
• CodeR: Issue Resolving with Multi-Agent and Task Graphs
  3 Jun 2024 1 repository listed
  GitHub issue resolving recently has attracted significant attention from academia and industry.
• Unraveling Code Clone Dynamics in Deep Learning Frameworks
  25 Apr 2024 1 repository listed
  We empirically analyze code clones in nine popular DL frameworks, i.
• Bug Characterization in Machine Learning-based Systems
  26 Jul 2023 1 repository listed
  Based on our results, fixing ML bugs are more costly and ML components are more error-prone, compared to non-ML bugs and non-ML components respectively.
• Automating Code-Related Tasks Through Transformers: The Impact of Pre-training
  8 Feb 2023 1 repository listed
  Then, we pre-train 32 transformers using both (i) generic pre-training objectives usually adopted in SE; and (ii) pre-training objectives tailored to specific code-related tasks subject of our experimentation, namely…
• Using Developer Discussions to Guide Fixing Bugs in Software
  11 Nov 2022 1 repository listed
  Automatically fixing software bugs is a challenging task.
• ADPTriage: Approximate Dynamic Programming for Bug Triage
  2 Nov 2022 1 repository listed
  In this study, we develop a Markov decision process (MDP) model for an online bug triage task.
• CoditT5: Pretraining for Source Code and Natural Language Editing
  10 Aug 2022 1 repository listed
  Pretrained language models have been shown to be effective in many software-related generation tasks; however, they are not well-suited for editing tasks as they are not designed to reason about edits.
• FixEval: Execution-based Evaluation of Program Fixes for Programming Problems
  15 Jun 2022 1 repository listed Syntology ran 6 of 14 samples · 8 unverified
  To address this issue, we introduce FixEval, a benchmark comprising of buggy code submissions to competitive programming problems and their corresponding fixes.
• S-DABT: Schedule and Dependency-Aware Bug Triage in Open-Source Bug Tracking Systems
  12 Apr 2022 1 repository listed
  We further incorporate the schedule of developers in our formulation to have a more comprehensive model for this multifaceted problem.
• RoPGen: Towards Robust Code Authorship Attribution via Automatic Coding Style Transformation
  12 Feb 2022 1 repository listed
  Recent studies show that current source code authorship attribution methods can be compromised by attackers exploiting adversarial examples and coding style manipulation.
• DABT: A Dependency-aware Bug Triaging Method
  26 Apr 2021 1 repository listed
  In software engineering practice, fixing a bug promptly reduces the associated costs.
• D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis
  16 Feb 2021 1 repository listed
  However, existing datasets to train models for vulnerability identification suffer from multiple limitations such as limited bug context, limited size, and synthetic and unrealistic source code.
• On the Embeddings of Variables in Recurrent Neural Networks for Source Code
  23 Oct 2020 1 repository listed
  In this work, we develop dynamic embeddings, a recurrent mechanism that adjusts the learned semantics of the variable when it obtains more information about the variable's role in the program.
• A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code
  23 Oct 2020 1 repository listed
  There is an emerging interest in the application of natural language processing models to source code processing tasks.
• Empirical Study of Transformers for Source Code
  15 Oct 2020 1 repository listed
  In this work, we conduct a thorough empirical study of the capabilities of Transformers to utilize syntactic information in different tasks.

Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.