Code Search

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

7 leaderboard tables shown for this task, 7 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

14 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 60 papers with code (125 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
  20 Sep 2019 14 repositories listed Syntology ran 8 of 17 samples · 9 unverified
  To enable evaluation of progress on code search, we are releasing the CodeSearchNet Corpus and are presenting the CodeSearchNet Challenge, which consists of 99 natural language queries with about 4k expert relevance…
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
  9 Feb 2021 7 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Benchmark datasets have a significant impact on accelerating research in programming language tasks.
• AutoCodeRover: Autonomous Program Improvement
  8 Apr 2024 5 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 1 pointer-only (licence)
  Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding.
• MELT: Mining Effective Lightweight Transformations from Pull Requests
  28 Aug 2023 2 repositories listed
  By leveraging code examples mined from the library source and automatically generated code examples based on the pull requests, we infer transformation rules in \comby, a language for structural code search and replace.
• Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data
  31 May 2023 2 repositories listed Syntology ran 5 of 8 samples · 3 unverified
  SANTA proposes two pretraining methods to make language models structure-aware and learn effective representations for structured data: 1) Structured Data Alignment, which utilizes the natural alignment relations…
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation
  13 May 2023 2 repositories listed Syntology ran 3 of 4 samples · 1 unverified
  To address these limitations, we propose ``CodeT5+'', a family of encoder-decoder LLMs for code in which component modules can be flexibly combined to suit a wide range of downstream code tasks.
• UniXcoder: Unified Cross-Modal Pre-training for Code Representation
  8 Mar 2022 2 repositories listed Syntology ran 2 of 4 samples · 2 unverified
  Furthermore, we propose to utilize multi-modal contents to learn representation of code fragment with contrastive learning, and then align representations among programming languages using a cross-modal generation task.
• Memorization and Generalization in Neural Code Intelligence Models
  16 Jun 2021 2 repositories listed
  The goal of this paper is to evaluate and compare the extent of memorization and generalization in neural code intelligence models.
• DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
  15 Feb 2021 2 repositories listed
  Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks.
• CoNCRA: A Convolutional Neural Network Code Retrieval Approach
  3 Sep 2020 2 repositories listed
  We propose a technique for semantic code search: A Convolutional Neural Network approach to code retrieval (CoNCRA).
• When Deep Learning Met Code Search
  9 May 2019 2 repositories listed
  Our evaluation shows that: 1.
• Zero-Shot Cross-Domain Code Search without Fine-Tuning
  10 Apr 2025 1 repository listed
  In this work, we propose to break the query-code matching process of code search into two simpler tasks: query-comment matching and code-code matching.
• Repository-level Code Search with Neural Retrieval Methods
  10 Feb 2025 1 repository listed
  We define the task of repository-level code search as retrieving the set of files from the current state of a code repository that are most relevant to addressing a user's question or bug.
• Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Search
  26 Nov 2024 1 repository listed
  Low isotropy in an embedding space impairs performance on tasks involving semantic inference.
• CodeSAM: Source Code Representation Learning by Infusing Self-Attention with Multi-Code-View Graphs
  21 Nov 2024 1 repository listed
  This progress is largely attributed to the development of generalizable source code representations that effectively capture the syntactic and semantic characteristics of code.
• ViC: Virtual Compiler Is All You Need For Assembly Code Search
  10 Aug 2024 1 repository listed
  Assembly code search is vital for reducing the burden on reverse engineers, allowing them to quickly identify specific functions using natural language within vast binary programs.
• CoIR: A Comprehensive Benchmark for Code Information Retrieval Models
  3 Jul 2024 1 repository listed Syntology ran 2 of 5 samples · 3 unverified
  Despite the substantial success of Information Retrieval (IR) in various NLP tasks, most IR systems predominantly handle queries and corpora in natural language, neglecting the domain of code retrieval.
• CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agents
  17 Jun 2024 1 repository listed
  Semantic code search, retrieving code that matches a given natural language query, is an important task to improve productivity in software engineering.
• RepoQA: Evaluating Long Context Code Understanding
  10 Jun 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  Recent advances have been improving the context windows of Large Language Models (LLMs).
• Advanced Detection of Source Code Clones via an Ensemble of Unsupervised Similarity Measures
  3 May 2024 1 repository listed
  The capability of accurately determining code similarity is crucial in many tasks related to software development.
• ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Search
  25 Mar 2024 1 repository listed
  Retrieval-based code question answering seeks to match user queries in natural language to relevant code snippets.
• Source Code Clone Detection Using Unsupervised Similarity Measures
  18 Jan 2024 1 repository listed
  Assessing similarity in source code has gained significant attention in recent years due to its importance in software engineering tasks such as clone detection and code search and recommendation.
• Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search
  9 Jan 2024 1 repository listed Syntology ran 5 of 9 samples · 4 unverified · 9 pointer-only (licence)
  In code search, the Generation-Augmented Retrieval (GAR) framework, which generates exemplar code snippets to augment queries, has emerged as a promising strategy to address the principal challenge of modality…
• GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understanding
  16 Nov 2023 1 repository listed
  Language models can serve as a valuable tool for software developers to increase productivity.
• TransformCode: A Contrastive Learning Framework for Code Embedding via Subtree Transformation
  10 Nov 2023 1 repository listed
  Our framework has several advantages over existing methods: (1) It is flexible and adaptable, because it can easily be extended to other downstream tasks that require code representation (such as code-clone detection…
• Rethinking Negative Pairs in Code Search
  12 Oct 2023 1 repository listed
  In our proposed loss function, we apply three methods to estimate the weights of negative pairs and show that the vanilla InfoNCE loss is a special case of Soft-InfoNCE.
• Language Models are Universal Embedders
  12 Oct 2023 1 repository listed
  As such cases span from English to other natural or programming languages, from retrieval to classification and beyond, it is desirable to build a unified embedding model rather than dedicated ones for each scenario.
• Constructing Multilingual Code Search Dataset Using Neural Machine Translation
  27 Jun 2023 1 repository listed
  Code search is a task to find programming codes that semantically match the given natural language queries.
• Backdooring Neural Code Search
  27 May 2023 1 repository listed Syntology ran 2 of 14 samples · 12 unverified
  Neural code search models are hence behind many such engines.
• The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation
  9 May 2023 1 repository listed
  We present The Vault, a dataset of high-quality code-text pairs in multiple programming languages for training large language models to understand and generate code.

Syntology lines on 10 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.