Datasets › CodeSearchNet

CodeSearchNet

Introduced by Hamel Husain et al. in CodeSearchNet Challenge: Evaluating the State of Semantic Code Search archive 2025-07-28

The CodeSearchNet Corpus is a large dataset of functions with associated documentation written in Go, Java, JavaScript, PHP, Python, and Ruby from open source projects on GitHub. The CodeSearchNet Corpus includes: * Six million methods overall * Two million of which have associated documentation (docstrings, JavaDoc, and more) * Metadata that indicates the original location (repository or line number, for example) where the data was found

Source: https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/

Benchmarks archive 2025-07-28

All 12 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

First row (archive order)PaperCode
Code Documentation Generation CodeSearchNet - Java CodeTrans-MT-Large Smoothed BLEU-4 21.87 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 8 Compare
Code Documentation Generation CodeSearchNet - Php CodeTrans-MT-Base Smoothed BLEU-4 26.23 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 8 Compare
Code Documentation Generation CodeSearchNet - JavaScript Transformer Smoothed BLEU-4 25.61 CodeBERT: A Pre-Trained Model for Programming and... salesforce/codet5 +8 8 Compare
Code Documentation Generation CodeSearchNet - Python CodeTrans-MT-Base Smoothed BLEU-4 20.39 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 7 Compare
Code Documentation Generation CodeSearchNet - Go CodeBERT (MLM) Smoothed BLEU-4 26.79 CodeBERT: A Pre-Trained Model for Programming and... salesforce/codet5 +8 7 Compare
Code Documentation Generation CodeSearchNet - Ruby CodeTrans-MT-Base Smoothed BLEU-4 15.26 CodeTrans: Towards Cracking the Language of Silicon's... agemagician/CodeTrans 7 Compare
Code Documentation Generation CodeSearchNet CodeBERT (MLM+RTD) Smoothed BLEU-4 15.99 CodeBERT: A Pre-Trained Model for Programming and... salesforce/codet5 +8 7 Compare
Code Search CodeSearchNet cpt-code M Overall 93.5 Text and Code Embeddings by Contrastive Pre-Training openmatch/coco-dr 6 Compare
Code Search CodeSearchNet - Ruby Uni-SBT MRR 0.3639 Multimodal Representation for Neural Code Search jianguda/mrncs 1 Compare
Source Code Summarization CodeSearchNet ContraCode F1 17.24 Contrastive Code Representation Learning parasj/contracode 1 Compare
Method name prediction CodeSearchNet ContraCode F1 17.24 Contrastive Code Representation Learning parasj/contracode 1 Compare
Source Code Summarization CodeSearchNet - Python AdaMo-basic BLEU-4 16.46 Assemble Foundation Models for Automatic Code Summarization jianguda/afm4acs 1 Compare

Papers archive 2025-07-28

papers with a benchmark row: 8 · with a code link: 8 · where Syntology ran a sample: 2 (2 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (2 of 8 with a benchmark row: 2 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 8 of 8 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 308. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

Dataset loaders archive 2025-07-28

4 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

Custom

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • CodeSearchNet
  • CodeSearchNet - Python
  • CodeSearchNet - Java
  • CodeSearchNet - Go
  • CodeSearchNet - Php
  • CodeSearchNet - Ruby
  • CodeSearchNet - JavaScript

7 variant names, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections