{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-unsupervised-dense-information","title":"Unsupervised Dense Information Retrieval with Contrastive Learning","arxiv_id":"2112.09118","date":"2021-12-16","proceeding":null,"authors":["Gautier Izacard","Mathilde Caron","Lucas Hosseini","Sebastian Riedel","Piotr Bojanowski","Armand Joulin","Edouard Grave"],"abstract":"Recently, information retrieval has seen the emergence of dense retrievers, using neural networks, as an alternative to classical sparse methods based on term-frequency. These models have obtained state-of-the-art results on datasets and tasks where large training sets are available. However, they do not transfer well to new applications with no training data, and are outperformed by unsupervised term-frequency methods such as BM25. In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings. On the BEIR benchmark our unsupervised model outperforms BM25 on 11 out of 15 datasets for the Recall@100. When used as pre-training before fine-tuning, either on a few thousands in-domain examples or on the large MS~MARCO dataset, our contrastive model leads to improvements on the BEIR benchmark. Finally, we evaluate our approach for multi-lingual retrieval, where training data is even scarcer than for English, and show that our approach leads to strong unsupervised performance. Our model also exhibits strong cross-lingual transfer when fine-tuned on supervised English data only and evaluated on low resources language such as Swahili. We show that our unsupervised models can perform cross-lingual retrieval between different scripts, such as retrieving English documents from Arabic queries, which would not be possible with term matching methods.","url_abs":"https://arxiv.org/abs/2112.09118v4","url_pdf":"https://arxiv.org/pdf/2112.09118v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-unsupervised-dense-information","repo_url":"https://github.com/facebookresearch/contriever","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"towards-unsupervised-dense-information","repo_url":"https://github.com/DevSinghSachan/unsupervised-passage-reranking","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"towards-unsupervised-dense-information","repo_url":"https://github.com/nthakur20/income","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-unsupervised-dense-information","repo_url":"https://github.com/soldni/pyterrier_sentence_transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-unsupervised-dense-information","repo_url":"https://github.com/thakur-nandan/income","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-unsupervised-dense-information","repo_url":"https://github.com/xfactlab/emnlp2023-damaging-retrieval","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"cross-lingual-transfer","task_name":"Cross-Lingual Transfer"},{"task_slug":"fact-checking","task_name":"Fact Checking"},{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"passage-retrieval","task_name":"Passage Retrieval"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"sentence-retrieval","task_name":"Sentence Retrieval"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/passage-retrieval-on-peerqa","task":"Passage Retrieval","dataset":"PeerQA","model":"Contriever-MS","rank_in_archive_order":6,"of":8,"metrics":{"MRR":"0.4408","Recall@10":"0.6314"},"uses_additional_data":false},{"leaderboard":"/sota/passage-retrieval-on-peerqa","task":"Passage Retrieval","dataset":"PeerQA","model":"Contriever","rank_in_archive_order":8,"of":8,"metrics":{"MRR":"0.3624","Recall@10":"0.5567"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.09118","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.09118"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xfactlab/emnlp2023-damaging-retrieval","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DevSinghSachan/unsupervised-passage-reranking","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nthakur20/income","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/soldni/pyterrier_sentence_transformers","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/contriever","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thakur-nandan/income","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":4},"by_repo_kind":{"listed":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b201d8d93b242b70","entry":"pad_input_ids","repo":"nthakur20/income","repo_kind":"listed","path":"income/jpq/preprocess.py","file_url":"https://github.com/nthakur20/income/blob/HEAD/income/jpq/preprocess.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b201d8d93b242b70"}},{"code_sha256_prefix":"0ffcad8ea0548ac6","entry":"PassagePreprocessingFn","repo":"nthakur20/income","repo_kind":"listed","path":"income/jpq/preprocess.py","file_url":"https://github.com/nthakur20/income/blob/HEAD/income/jpq/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ffcad8ea0548ac6"}},{"code_sha256_prefix":"a9a03792761f628c","entry":"get_collate_function","repo":"nthakur20/income","repo_kind":"listed","path":"income/jpq/dataset.py","file_url":"https://github.com/nthakur20/income/blob/HEAD/income/jpq/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9a03792761f628c"}},{"code_sha256_prefix":"7cc208c6b8263844","entry":"multi_file_process","repo":"nthakur20/income","repo_kind":"listed","path":"income/jpq/preprocess.py","file_url":"https://github.com/nthakur20/income/blob/HEAD/income/jpq/preprocess.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7cc208c6b8263844"}},{"code_sha256_prefix":"7f738a55dde7b691","entry":"pack_tensor_2D","repo":"nthakur20/income","repo_kind":"listed","path":"income/jpq/dataset.py","file_url":"https://github.com/nthakur20/income/blob/HEAD/income/jpq/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f738a55dde7b691"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}