{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/corpusbrain-pre-train-a-generative-retrieval","title":"CorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks","arxiv_id":"2208.07652","date":"2022-08-16","proceeding":null,"authors":["Jiangui Chen","Ruqing Zhang","Jiafeng Guo","Yiqun Liu","Yixing Fan","Xueqi Cheng"],"abstract":"Knowledge-intensive language tasks (KILT) usually require a large body of information to provide correct answers. A popular paradigm to solve this problem is to combine a search system with a machine reader, where the former retrieves supporting evidences and the latter examines them to produce answers. Recently, the reader component has witnessed significant advances with the help of large-scale pre-trained generative models. Meanwhile most existing solutions in the search component rely on the traditional ``index-retrieve-then-rank'' pipeline, which suffers from large memory footprint and difficulty in end-to-end optimization. Inspired by recent efforts in constructing model-based IR models, we propose to replace the traditional multi-step search pipeline with a novel single-step generative model, which can dramatically simplify the search process and be optimized in an end-to-end manner. We show that a strong generative retrieval model can be learned with a set of adequately designed pre-training tasks, and be adopted to improve a variety of downstream KILT tasks with further fine-tuning. We name the pre-trained generative retrieval model as CorpusBrain as all information about the corpus is encoded in its parameters without the need of constructing additional index. Empirical results show that CorpusBrain can significantly outperform strong baselines for the retrieval task on the KILT benchmark and establish new state-of-the-art downstream performances. We also show that CorpusBrain works well under zero- and low-resource settings.","url_abs":"https://arxiv.org/abs/2208.07652v1","url_pdf":"https://arxiv.org/pdf/2208.07652v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"corpusbrain-pre-train-a-generative-retrieval","repo_url":"https://github.com/ict-bigdatalab/corpusbrain","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2208.07652","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2208.07652"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ict-bigdatalab/corpusbrain","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d43a426d78c2dacb","entry":"chunk_it","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"genre/utils.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/genre/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d43a426d78c2dacb"}},{"code_sha256_prefix":"e7e75981cb464788","entry":"normalize_answer","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"kilt/eval_downstream.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/kilt/eval_downstream.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7e75981cb464788"}},{"code_sha256_prefix":"6d7cebaccb13bd3c","entry":"create_input","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"genre/utils.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/genre/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6d7cebaccb13bd3c"}},{"code_sha256_prefix":"a0b34bcb573fbd1c","entry":"get_entity_spans_pre_processing","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"genre/utils.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/genre/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a0b34bcb573fbd1c"}},{"code_sha256_prefix":"1822840bf1a15385","entry":"get_gold_answers","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"kilt/eval_downstream.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/kilt/eval_downstream.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1822840bf1a15385"}},{"code_sha256_prefix":"e7e608263bcd02c2","entry":"get_gold_titles","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"kilt/eval_downstream.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/kilt/eval_downstream.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7e608263bcd02c2"}},{"code_sha256_prefix":"6a9bcdcb79e942e6","entry":"get_rank","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"kilt/eval_retrieval.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/kilt/eval_retrieval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a9bcdcb79e942e6"}},{"code_sha256_prefix":"8127c3fe61cc2993","entry":"rprecision","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"kilt/eval_retrieval.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/kilt/eval_retrieval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8127c3fe61cc2993"}},{"code_sha256_prefix":"1a66d75f4a44d89f","entry":"run_thread","repo":"ict-bigdatalab/corpusbrain","repo_kind":"official","path":"kilt/dataset_mapper.py","file_url":"https://github.com/ict-bigdatalab/corpusbrain/blob/HEAD/kilt/dataset_mapper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1a66d75f4a44d89f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}