Papers › Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intent

Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intent

27 Aug 2020arXiv:2008.12193archive 2025-07-28

Geert Heyman, Tom Van Cutsem

In this work, we propose and study annotated code search: the retrieval of code snippets paired with brief descriptions of their intent using natural language queries. On three benchmark datasets, we investigate how code retrieval systems can be improved by leveraging descriptions to better capture the intents of code snippets. Building on recent progress in transfer learning and natural language processing, we create a domain-specific retrieval model for code annotated with a natural language description. We find that our model yields significantly more relevant search results (with absolute gains up to 20.6% in mean reciprocal rank) compared to state-of-the-art code retrieval methods that do not use descriptions but attempt to compute the intent of snippets solely from unannotated code.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

nokia/codesearch officialmentioned in papermentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Annotated Code SearchCode SearchInformation RetrievalNatural Language QueriesRetrievalTransfer Learning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Annotated Code Search PACS-CoNaLa Ensemble:USE-tuned+NCS MRR 0.351 #1 of 4 Archive leaderboard report
Annotated Code Search PACS-CoNaLa USE-tuned MRR 0.340 #2 of 4 Archive leaderboard report
Annotated Code Search PACS-CoNaLa USE MRR 0.181 #3 of 4 Archive leaderboard report
Annotated Code Search PACS-CoNaLa NCS MRR 0.167 #4 of 4 Archive leaderboard report
Annotated Code Search PACS-SO-DS Ensemble:USE-tuned+NCS MRR 0.323 #1 of 4 Archive leaderboard report
Annotated Code Search PACS-SO-DS USE-tuned MRR 0.304 #2 of 4 Archive leaderboard report
Annotated Code Search PACS-SO-DS USE MRR 0.244 #3 of 4 Archive leaderboard report
Annotated Code Search PACS-SO-DS NCS MRR 0.113 #4 of 4 Archive leaderboard report
Annotated Code Search PACS-StaQC-py Ensemble:USE-tuned+NCS MRR 0.126 #1 of 4 Archive leaderboard report
Annotated Code Search PACS-StaQC-py USE-tuned MRR 0.117 #2 of 4 Archive leaderboard report
Annotated Code Search PACS-StaQC-py USE MRR 0.104 #3 of 4 Archive leaderboard report
Annotated Code Search PACS-StaQC-py NCS MRR 0.030 #4 of 4 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections