{"url":"/dataset/defects4j","name":"Defects4J","full_name":"Defects4J","description_markdown":"Defects4J is a collection of reproducible bugs and a supporting infrastructure with the goal of advancing software engineering research.\r\n\r\nDefects4J contains 835 bugs (plus 29 deprecated bugs) from the following open-source projects:\r\n\r\n| Identifier      | Project name               | Number of active bugs | Active bug ids      | Deprecated bug ids (\\*) |\r\n|-----------------|----------------------------|----------------------:|---------------------|-------------------------| \r\n| Chart           | jfreechart                 |           26          | 1-26                | None                    |\r\n| Cli             | commons-cli                |           39          | 1-5,7-40            | 6                       |\r\n| Closure         | closure-compiler           |          174          | 1-62,64-92,94-176   | 63,93                   |\r\n| Codec           | commons-codec              |           18          | 1-18                | None                    |\r\n| Collections     | commons-collections        |            4          | 25-28               | 1-24                    |\r\n| Compress        | commons-compress           |           47          | 1-47                | None                    |\r\n| Csv             | commons-csv                |           16          | 1-16                | None                    |\r\n| Gson            | gson                       |           18          | 1-18                | None                    |\r\n| JacksonCore     | jackson-core               |           26          | 1-26                | None                    |\r\n| JacksonDatabind | jackson-databind           |          112          | 1-112               | None                    |\r\n| JacksonXml      | jackson-dataformat-xml     |            6          | 1-6                 | None                    |\r\n| Jsoup           | jsoup                      |           93          | 1-93                | None                    |\r\n| JxPath          | commons-jxpath             |           22          | 1-22                | None                    |\r\n| Lang            | commons-lang               |           64          | 1,3-65              | 2                       |\r\n| Math            | commons-math               |          106          | 1-106               | None                    |\r\n| Mockito         | mockito                    |           38          | 1-38                | None                    |\r\n| Time            | joda-time                  |           26          | 1-20,22-27          | 21                      |","description_withheld":null,"homepage":"https://github.com/rjust/defects4j","introduced_date":"2014-07-01","introduced_date_note":null,"introduced_by":null,"license":{"name":"MIT License","url":"https://opensource.org/license/mit/"},"modalities":[],"tasks":[{"name":"Code Completion","url":"/task/code-completion","datasets_with_task":"/datasets/task/code-completion"},{"name":"Program Repair","url":"/task/program-repair","datasets_with_task":"/datasets/task/program-repair"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["Defects4J"],"data_loaders":[],"num_papers_in_archive":9,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/code-completion-on-defects4j","task":"Code Completion","dataset_variant":"Defects4J","rows":2,"metrics":["Compilation Rate","Pass@1","BLEU"],"first_row_in_archive_order":{"model":"Rambo","paper":"/paper/rambo-enhancing-rag-based-repository-level","metrics":{"BLEU":"66.29","Compilation Rate":"76.47","Pass@1":"63.73"},"code_links":[{"title":"ise-uet-vnu/rambo","url":"https://github.com/ise-uet-vnu/rambo"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/rambo-enhancing-rag-based-repository-level","title":"RAMBO: Enhancing RAG-based Repository-Level Method Body Completion","date":"2024-09-23","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/repocoder-repository-level-code-completion","title":"RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation","date":"2023-03-22","rows_on_this_dataset":1,"code_links":1,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":7,"samples_ran":1,"samples_unverified":6,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":1,"samples_harvested":7,"samples_ran":1,"samples_unverified":6,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}