{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-the-role-of-token-retrieval-in","title":"Rethinking the Role of Token Retrieval in Multi-Vector Retrieval","arxiv_id":"2304.01982","date":"2023-04-04","proceeding":"NeurIPS 2023 11","authors":["Jinhyuk Lee","Zhuyun Dai","Sai Meher Karthik Duddu","Tao Lei","Iftekhar Naim","Ming-Wei Chang","Vincent Y. Zhao"],"abstract":"Multi-vector retrieval models such as ColBERT [Khattab and Zaharia, 2020] allow token-level interactions between queries and documents, and hence achieve state of the art on many information retrieval benchmarks. However, their non-linear scoring function cannot be scaled to millions of documents, necessitating a three-stage process for inference: retrieving initial candidates via token retrieval, accessing all token vectors, and scoring the initial candidate documents. The non-linear scoring function is applied over all token vectors of each candidate document, making the inference process complicated and slow. In this paper, we aim to simplify the multi-vector retrieval by rethinking the role of token retrieval. We present XTR, ConteXtualized Token Retriever, which introduces a simple, yet novel, objective function that encourages the model to retrieve the most important document tokens first. The improvement to token retrieval allows XTR to rank candidates only using the retrieved tokens rather than all tokens in the document, and enables a newly designed scoring stage that is two-to-three orders of magnitude cheaper than that of ColBERT. On the popular BEIR benchmark, XTR advances the state-of-the-art by 2.8 nDCG@10 without any distillation. Detailed analysis confirms our decision to revisit the token retrieval stage, as XTR demonstrates much better recall of the token retrieval stage compared to ColBERT.","url_abs":"https://arxiv.org/abs/2304.01982v3","url_pdf":"https://arxiv.org/pdf/2304.01982v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-the-role-of-token-retrieval-in","repo_url":"https://github.com/google-deepmind/xtr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"rethinking-the-role-of-token-retrieval-in","repo_url":"https://github.com/primeqa/primeqa/blob/xtr/notebooks/ir/dense/xtr_retrieval.ipynb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.01982","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.01982"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/primeqa/primeqa/blob/xtr/notebooks/ir/dense/xtr_retrieval.ipynb","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/xtr","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ef780083766d9462","entry":"get_language_id","repo":"primeqa/primeqa","repo_kind":"listed","path":"primeqa/ir/run_bm25_retrieval.py","file_url":"https://github.com/primeqa/primeqa/blob/HEAD/primeqa/ir/run_bm25_retrieval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef780083766d9462"}},{"code_sha256_prefix":"bcf98d7b2fc80e5e","entry":"load_queries","repo":"primeqa/primeqa","repo_kind":"listed","path":"primeqa/ir/run_bm25_retrieval.py","file_url":"https://github.com/primeqa/primeqa/blob/HEAD/primeqa/ir/run_bm25_retrieval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bcf98d7b2fc80e5e"}},{"code_sha256_prefix":"316cd1c0ecccd603","entry":"object_reference","repo":"primeqa/primeqa","repo_kind":"listed","path":"primeqa/calibration/train_confidence_calibrator.py","file_url":"https://github.com/primeqa/primeqa/blob/HEAD/primeqa/calibration/train_confidence_calibrator.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"316cd1c0ecccd603"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}