{"url":"/dataset/complexcodeeval","name":"ComplexCodeEval","full_name":null,"description_markdown":"# ComplexCodeEval\r\n\r\nComplexCodeEval is an evaluation benchmark designed to accommodate multiple downstream tasks, accurately reflect different programming environments, and deliberately avoid data leakage issues. This benchmark includes a diverse set of samples from real-world projects, aiming to closely mirror actual development scenarios.\r\n\r\n## Overview\r\n\r\nComplexCodeEval consists of:\r\n- **3,897 Java samples** from **1,055 code repositories**\r\n- **7,184 Python samples** from **2,107 code repositories**\r\n\r\n### Key Features\r\n\r\n1. **Diverse Downstream Tasks**: The benchmark supports multiple downstream tasks to evaluate the performance of different code analysis tools and models.\r\n2. **Accurate Reflection of Programming Environments**: Samples are selected from projects that use popular third-party frameworks and packages.\r\n3. **Avoidance of Data Leakage**: Incorporates multiple timestamps for each sample to prevent data leakage.","description_withheld":null,"homepage":"","introduced_date":"2024-09-16","introduced_date_note":null,"introduced_by":{"paper":"/paper/complexcodeeval-a-benchmark-for-evaluating","title":"ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code","first_author":null,"url":null},"license":null,"modalities":[],"tasks":[],"languages":[],"variants":["ComplexCodeEval"],"data_loaders":[],"num_papers_in_archive":5,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}