Datasets › RepoBench
RepoBench
RepoBench is a benchmark designed for evaluating repository-level code auto-completion systems, focusing on more complex, real-world programming scenarios involving multiple files. It comprises three tasks: RepoBench-R (Retrieval), measuring the system's ability to retrieve relevant code snippets; RepoBench-C (Code Completion), assessing the prediction of the next line of code with both in-file and cross-file context; and RepoBench-P (Pipeline), evaluating complex tasks requiring both retrieval and prediction. RepoBench aims to provide a comprehensive performance comparison to foster continuous improvement in auto-completion systems.
Benchmarks archive 2025-07-28
No leaderboard in the archive resolves to this dataset.
Papers archive 2025-07-28
No paper in the archive has a leaderboard row on this dataset; the archive counts 17 papers for it but never published that list.
Dataset loaders archive 2025-07-28
No loader listed in the archive.
Tasks archive 2025-07-28
No task tagged in the archive.
License archive 2025-07-28
No licence recorded in the archive. Absence here is not a statement about the dataset's terms.
Modalities archive 2025-07-28
No modality tagged.
Languages archive 2025-07-28
No language tagged.
Variants archive 2025-07-28
- RepoBench
1 variant name, as the archive lists them.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections