Datasets › CodeXGLUE
CodeXGLUE
CodeXGLUE is a benchmark dataset and open challenge for code intelligence. It includes a collection of code intelligence tasks and a platform for model evaluation and comparison. CodeXGLUE stands for General Language Understanding Evaluation benchmark for CODE. It includes 14 datasets for 10 diversified code intelligence tasks covering the following scenarios:
- code-code (clone detection, defect detection, cloze test, code completion, code repair, and code-to-code translation)
- text-code (natural language code search, text-to-code generation)
- code-text (code summarization)
- text-text (documentation translation)
A brief summary of CodeXGLUE is provided in the figure, including tasks, datasets, language, sizes in various states, baseline systems, providers, and short definitions of each task. Datasets highlighted in BLUE are newly introduced.
Image source: https://github.com/microsoft/CodeXGLUE
Benchmarks archive 2025-07-28
All 10 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
Papers archive 2025-07-28
6 shown of 6 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 205. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.
| Date | Samples run Syntology | |||
|---|---|---|---|---|
| CodeT5+: Open Code Large Language Models for Code Understanding and Generation | 2 | 6 | 13 May 2023 | ran 3 of 4 samples (1 unverified) |
| Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar | 0 | 1 | 13 Apr 2022 | not harvested |
| GAP-Gen: Guided Automatic Python Code Generation | 1 | 1 | 19 Jan 2022 | not harvested |
| CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation | 5 | 2 | 2 Sep 2021 | ran 1 of 11 samples (10 unverified) |
| Retrieval Augmented Code Generation and Summarization | 2 | 1 | 26 Aug 2021 | ran 2 of 3 samples (1 unverified; 3 pointer-only for licence) |
| CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation | 7 | 9 | 9 Feb 2021 | ran 1 of 1 samples (0 unverified) |
Dataset loaders archive 2025-07-28
No loader listed in the archive.
Tasks archive 2025-07-28
License archive 2025-07-28
Computational Use of Data Agreement (C-UDA) License
Modalities archive 2025-07-28
Languages archive 2025-07-28
No language tagged.
Variants archive 2025-07-28
- CodeXGLUE - WebQueryTest
- CodeXGLUE - PY150
- CodeXGLUE - POJ-104
- CodeXGLUE - Microsoft Docs
- CodeXGLUE - Github Java Corpus
- CodeXGLUE - Devign
- CodeXGLUE - CT-maxmin
- CodeXGLUE - CT-all
- CodeXGLUE - CONCODE
- CodeXGLUE - CodeTrans
- CodeXGLUE - CodeSearchNet
- CodeXGLUE - Bugs2Fix
- CodeXGLUE - AdvTest
- CodeXGLUE - BigCloneBench
- CodeXGLUE
15 variant names, as the archive lists them.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections