{"url":"/dataset/apibench","name":"APIBench","full_name":null,"description_markdown":"**APIBench** is a benchmark dataset designed for evaluating the performance of **API recommendation approaches**. It was introduced in the paper titled \"Revisiting, Benchmarking, and Exploring API Recommendation: How Far Are We?\"¹. Let's delve into the details:\r\n\r\n1. **Purpose and Context**:\r\n   - **APIBench** serves as a standardized evaluation platform for assessing the effectiveness of various API recommendation techniques.\r\n   - Researchers and practitioners can use it to compare and analyze different approaches in the field of API recommendation.\r\n\r\n2. **Components**:\r\n   - **APIBench** comprises two sub-datasets:\r\n     - **APIBench-Q**: This dataset focuses on **query-based API recommendation**. It involves providing relevant APIs to developers based on natural language queries that describe programming requirements.\r\n     - **APIBench-C**: This dataset deals with **code-based API recommendation**. It aims to predict the next API to use based on the code context.\r\n\r\n3. **Contents**:\r\n   - **APIBench-Q**:\r\n     - Contains both **Java** and **Python** versions.\r\n     - Includes original queries along with corresponding APIs and API classes.\r\n     - Also provides reformulated queries derived from the original ones using various techniques (such as RACK, NLP2API, SEQUER, Google Prediction Service, and NLPAUG).\r\n     - The dataset size:\r\n       - Original Queries:\r\n         - Python: 4,309\r\n         - Java: 6,563\r\n       - Expanded Queries (after reformulation):\r\n         - Python: 173,517\r\n         - Java: 400,126\r\n\r\n   - **APIBench-C**:\r\n     - Similar to **APIBench-Q**, it has both **Java** and **Python** versions.\r\n     - Contains metadata files for code in different domains.\r\n     - Metadata includes information about function lengths (long, normal, and short keywords).\r\n\r\n4. **Access**:\r\n   - Since GitHub does not host large datasets, you can download the **APIBench** dataset from **Zenodo**².\r\n\r\nIn summary, **APIBench** provides a valuable resource for evaluating and advancing API recommendation techniques, fostering research and development in this area. 🚀🔍\r\n\r\nSource: Conversation with Bing, 3/20/2024\r\n(1) GitHub - JohnnyPeng18/APIBench: APIBench is a benchmark for evaluating .... https://github.com/JohnnyPeng18/APIBench.\r\n(2) APIBench: A Benchmark Dataset for Evaluating API ... - Zenodo. https://zenodo.org/records/5797297.\r\n(3) api-benchmark - npm. https://www.npmjs.com/package/api-benchmark.\r\n(4) The world's #1 API & code benchmark platform | CyBench. https://cybench.io/.\r\n(5) undefined. https://github.com/masud-technope/RACK-Replication-Package.\r\n(6) undefined. https://github.com/masud-technope/NLP2API-Replication-Package.\r\n(7) undefined. https://github.com/kbcao/sequer.\r\n(8) undefined. http://suggestqueries.google.com/complete/search?.\r\n(9) undefined. https://github.com/makcedward/nlpaug.","description_withheld":null,"homepage":"https://github.com/JohnnyPeng18/APIBench","introduced_date":"2021-12-23","introduced_date_note":null,"introduced_by":{"paper":"/paper/revisiting-benchmarking-and-exploring-api","title":"Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We?","first_author":null,"url":null},"license":null,"modalities":[],"tasks":[],"languages":[],"variants":["APIBench"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}