APIBench

APIBench is a benchmark dataset designed for evaluating the performance of API recommendation approaches. It was introduced in the paper titled "Revisiting, Benchmarking, and Exploring API Recommendation: How Far Are We?"¹. Let's delve into the details:

1. Purpose and Context:
2. APIBench serves as a standardized evaluation platform for assessing the effectiveness of various API recommendation techniques.

3. Researchers and practitioners can use it to compare and analyze different approaches in the field of API recommendation.

4. Components:

5. APIBench comprises two sub-datasets:

   • APIBench-Q: This dataset focuses on query-based API recommendation. It involves providing relevant APIs to developers based on natural language queries that describe programming requirements.
   • APIBench-C: This dataset deals with code-based API recommendation. It aims to predict the next API to use based on the code context.

6. Contents:

7. APIBench-Q:

   • Contains both Java and Python versions.
   • Includes original queries along with corresponding APIs and API classes.
   • Also provides reformulated queries derived from the original ones using various techniques (such as RACK, NLP2API, SEQUER, Google Prediction Service, and NLPAUG).
   • The dataset size:
   • Original Queries:
     • Python: 4,309
     • Java: 6,563
   • Expanded Queries (after reformulation):
     • Python: 173,517
     • Java: 400,126

8. APIBench-C:

   • Similar to APIBench-Q, it has both Java and Python versions.
   • Contains metadata files for code in different domains.
   • Metadata includes information about function lengths (long, normal, and short keywords).

9. Access:

10. Since GitHub does not host large datasets, you can download the APIBench dataset from Zenodo².

In summary, APIBench provides a valuable resource for evaluating and advancing API recommendation techniques, fostering research and development in this area. 🚀🔍

Source: Conversation with Bing, 3/20/2024 (1) GitHub - JohnnyPeng18/APIBench: APIBench is a benchmark for evaluating .... https://github.com/JohnnyPeng18/APIBench. (2) APIBench: A Benchmark Dataset for Evaluating API ... - Zenodo. https://zenodo.org/records/5797297. (3) api-benchmark - npm. https://www.npmjs.com/package/api-benchmark. (4) The world's #1 API & code benchmark platform | CyBench. https://cybench.io/. (5) undefined. https://github.com/masud-technope/RACK-Replication-Package. (6) undefined. https://github.com/masud-technope/NLP2API-Replication-Package. (7) undefined. https://github.com/kbcao/sequer. (8) undefined. http://suggestqueries.google.com/complete/search?. (9) undefined. https://github.com/makcedward/nlpaug.