Papers › Text and Code Embeddings by Contrastive Pre-Training

Text and Code Embeddings by Contrastive Pre-Training

24 Jan 2022arXiv:2201.10005archive 2025-07-28

Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, Peter Welinder, Lilian Weng

Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use cases, varying in dataset choice, training objective and model architecture. In this work, we show that contrastive pre-training on unsupervised data at scale leads to high quality vector representations of text and code. The same unsupervised text embeddings that achieve new state-of-the-art results in linear-probe classification also display impressive semantic search capabilities and sometimes even perform competitively with fine-tuned models. On linear-probe classification accuracy averaging over 7 tasks, our best unsupervised model achieves a relative improvement of 4% and 1.8% over previous best unsupervised and supervised text embedding models respectively. The same text embeddings when evaluated on large-scale semantic search attains a relative improvement of 23.4%, 14.7%, and 10.6% over previous best unsupervised methods on MSMARCO, Natural Questions and TriviaQA benchmarks, respectively. Similarly to text embeddings, we train code embedding models on (text, code) pairs, obtaining a 20.8% relative improvement over prior best work on code search.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

openmatch/coco-dr mentioned on GitHubpytorchMIT report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Code SearchLinear-Probe ClassificationNatural QuestionsPassage RankingTriviaQAZero-shot Text Searchtext similarity

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Code Search CodeSearchNet cpt-code M Go 97.5 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code M JS 86.5 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code M Java 94.4 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code M Overall 93.5 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code M PHP 97.2 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code M Python 99.9 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code M Ruby 85.5 #1 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S Go 97.7 #2 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S JS 86.0 #2 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S Java 94.0 #2 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S Overall 93.4 #2 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S PHP 96.7 #2 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S Python 99.8 #2 of 6 Archive leaderboard report
Code Search CodeSearchNet cpt-code S Ruby 86.3 #2 of 6 Archive leaderboard report
Passage Ranking MS MARCO Fine-tuned SOTA MRR@10 44.3 #1 of 4 Archive leaderboard report
Passage Ranking MS MARCO cpt-text XL MRR@10 22.7 #2 of 4 Archive leaderboard report
Passage Ranking MS MARCO cpt-text L MRR@10 21.5 #3 of 4 Archive leaderboard report
Passage Ranking MS MARCO BM25 MRR@10 18.4 #4 of 4 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections