Datasets › CodeSearchNet

CodeSearchNet

Introduced by Hamel Husain et al. in CodeSearchNet Challenge: Evaluating the State of Semantic Code Search archive 2025-07-28

The CodeSearchNet Corpus is a large dataset of functions with associated documentation written in Go, Java, JavaScript, PHP, Python, and Ruby from open source projects on GitHub. The CodeSearchNet Corpus includes: * Six million methods overall * Two million of which have associated documentation (docstrings, JavaDoc, and more) * Metadata that indicates the original location (repository or line number, for example) where the data was found

Source: https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/

Benchmarks archive 2025-07-28

All 12 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

First row (archive order)PaperCode
Code Documentation Generation CodeSearchNet - Java CodeTrans-MT-Large Smoothed BLEU-4 21.87 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 8 Compare
Code Documentation Generation CodeSearchNet - Php CodeTrans-MT-Base Smoothed BLEU-4 26.23 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 8 Compare
Code Documentation Generation CodeSearchNet - JavaScript Transformer Smoothed BLEU-4 25.61 CodeBERT: A Pre-Trained Model for Programming and... salesforce/codet5 +8 8 Compare
Code Documentation Generation CodeSearchNet - Python CodeTrans-MT-Base Smoothed BLEU-4 20.39 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 7 Compare
Code Documentation Generation CodeSearchNet - Go CodeBERT (MLM) Smoothed BLEU-4 26.79 CodeBERT: A Pre-Trained Model for Programming and... salesforce/codet5 +8 7 Compare
Code Documentation Generation CodeSearchNet - Ruby CodeTrans-MT-Base Smoothed BLEU-4 15.26 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 7 Compare
Code Documentation Generation CodeSearchNet CodeBERT (MLM+RTD) Smoothed BLEU-4 15.99 CodeBERT: A Pre-Trained Model for Programming and... salesforce/codet5 +8 7 Compare
Code Search CodeSearchNet cpt-code M Overall 93.5 Text and Code Embeddings by Contrastive Pre-Training openmatch/coco-dr 6 Compare
Code Search CodeSearchNet - Ruby Uni-SBT MRR 0.3639 Multimodal Representation for Neural Code Search jianguda/mrncs 1 Compare
Source Code Summarization CodeSearchNet ContraCode F1 17.24 Contrastive Code Representation Learning parasj/contracode 1 Compare
Method name prediction CodeSearchNet ContraCode F1 17.24 Contrastive Code Representation Learning parasj/contracode 1 Compare
Source Code Summarization CodeSearchNet - Python AdaMo-basic BLEU-4 16.46 Assemble Foundation Models for Automatic Code Summarization jianguda/afm4acs 1 Compare

Papers archive 2025-07-28

8 shown of 8 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 308. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.

DateSamples run Syntology
CodeT5+: Open Code Large Language Models for Code Understanding and Generation 2 2 13 May 2023 ran 3 of 4 samples (1 unverified)
Text and Code Embeddings by Contrastive Pre-Training 1 2 24 Jan 2022 not harvested
Assemble Foundation Models for Automatic Code Summarization 1 1 13 Jan 2022 not harvested
Multimodal Representation for Neural Code Search 1 1 2 Jul 2021 not harvested
CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computing 1 6 6 Apr 2021 not harvested
GraphCodeBERT: Pre-training Code Representations with Data Flow 1 1 17 Sep 2020 not harvested
Contrastive Code Representation Learning 1 2 9 Jul 2020 not harvested
CodeBERT: A Pre-Trained Model for Programming and Natural Languages 9 47 19 Feb 2020 ran 2 of 20 samples (18 unverified; 4 pointer-only for licence)

Dataset loaders archive 2025-07-28

4 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

Custom

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • CodeSearchNet
  • CodeSearchNet - Python
  • CodeSearchNet - Java
  • CodeSearchNet - Go
  • CodeSearchNet - Php
  • CodeSearchNet - Ruby
  • CodeSearchNet - JavaScript

7 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections