Papers › Learning Dense Representations of Phrases at Scale

Learning Dense Representations of Phrases at Scale

23 Dec 2020ACL 2021 5arXiv:2012.12624archive 2025-07-28

Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, Danqi Chen

Open-domain question answering can be reformulated as a phrase retrieval problem, without the need for processing documents on-demand during inference (Seo et al., 2019). However, current phrase retrieval models heavily depend on sparse representations and still underperform retriever-reader approaches. In this work, we show for the first time that we can learn dense representations of phrases alone that achieve much stronger performance in open-domain QA. We present an effective method to learn phrase representations from the supervision of reading comprehension tasks, coupled with novel negative sampling methods. We also propose a query-side fine-tuning strategy, which can support transfer learning and reduce the discrepancy between training and inference. On five popular open-domain QA datasets, our model DensePhrases improves over previous phrase retrieval models by 15%-25% absolute accuracy and matches the performance of state-of-the-art retriever-reader models. Our model is easy to parallelize due to pure dense representations and processes more than 10 questions per second on CPUs. Finally, we directly use our pre-indexed dense phrase representations for two slot filling tasks, showing the promise of utilizing DensePhrases as a dense knowledge base for downstream tasks.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

jhyuklee/DensePhrases officialmentioned in papermentioned on GitHubpytorchSyntology: not harvested report
princeton-nlp/DensePhrases officialmentioned in papermentioned on GitHubpytorchApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Phrase Retrieval Learns Passage Retrieval, Too (arXiv:2109.08133; that paper's own run record: 11 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 2 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 8 unverified); nothing checks that they implement this paper's method. report
dmis-lab/gener mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ACL Anthology id 2022.emnlp-main.417 (that paper's own run record: 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
princeton-nlp/SimCSE mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via... (arXiv:2601.21218; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Open-Domain Question AnsweringQuestion AnsweringQuestion GenerationReading ComprehensionRetrievalSlot FillingTransfer Learning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Question Answering Natural Questions (long) DensePhrases EM 71.9 #1 of 13 Archive leaderboard report
Question Answering Natural Questions (long) DensePhrases F1 79.6 #1 of 13 Archive leaderboard report
Question Answering SQuAD1.1 dev DensePhrases EM 78.3 #18 of 55 Archive leaderboard report
Question Answering SQuAD1.1 dev DensePhrases F1 86.3 #18 of 55 Archive leaderboard report
Slot Filling KILT: T-REx DensePhrases Accuracy 53.9 #8 of 20 Archive leaderboard report
Slot Filling KILT: T-REx DensePhrases F1 61.74 #8 of 20 Archive leaderboard report
Slot Filling KILT: T-REx DensePhrases KILT-AC 27.84 #8 of 20 Archive leaderboard report
Slot Filling KILT: T-REx DensePhrases KILT-F1 32.34 #8 of 20 Archive leaderboard report
Slot Filling KILT: T-REx DensePhrases R-Prec 37.62 #8 of 20 Archive leaderboard report
Slot Filling KILT: T-REx DensePhrases Recall@5 40.07 #8 of 20 Archive leaderboard report
Slot Filling KILT: Zero Shot RE DensePhrases Accuracy 47.42 #7 of 21 Archive leaderboard report
Slot Filling KILT: Zero Shot RE DensePhrases F1 54.75 #7 of 21 Archive leaderboard report
Slot Filling KILT: Zero Shot RE DensePhrases KILT-AC 41.34 #7 of 21 Archive leaderboard report
Slot Filling KILT: Zero Shot RE DensePhrases KILT-F1 46.79 #7 of 21 Archive leaderboard report
Slot Filling KILT: Zero Shot RE DensePhrases R-Prec 57.43 #7 of 21 Archive leaderboard report
Slot Filling KILT: Zero Shot RE DensePhrases Recall@5 60.47 #7 of 21 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections