{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-training-of-multi-document-reader","title":"End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering","arxiv_id":"2106.05346","date":"2021-06-09","proceeding":"NeurIPS 2021 12","authors":["Devendra Singh Sachan","Siva Reddy","William Hamilton","Chris Dyer","Dani Yogatama"],"abstract":"We present an end-to-end differentiable training method for retrieval-augmented open-domain question answering systems that combine information from multiple retrieved documents when generating answers. We model retrieval decisions as latent variables over sets of relevant documents. Since marginalizing over sets of retrieved documents is computationally hard, we approximate this using an expectation-maximization algorithm. We iteratively estimate the value of our latent variable (the set of relevant documents for a given question) and then use this estimate to update the retriever and reader parameters. We hypothesize that such end-to-end training allows training signals to flow to the reader and then to the retriever better than staged-wise training. This results in a retriever that is able to select more relevant documents for a question and a reader that is trained on more accurate documents to generate an answer. Experiments on three benchmark datasets demonstrate that our proposed method outperforms all existing approaches of comparable size by 2-3% absolute exact match points, achieving new state-of-the-art results. Our results also demonstrate the feasibility of learning to retrieve to improve answer generation without explicit supervision of retrieval decisions.","url_abs":"https://arxiv.org/abs/2106.05346v2","url_pdf":"https://arxiv.org/pdf/2106.05346v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-training-of-multi-document-reader","repo_url":"https://github.com/DevSinghSachan/emdr2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"end-to-end-training-of-multi-document-reader","repo_url":"https://github.com/DevSinghSachan/art","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"answer-generation","task_name":"Answer Generation"},{"task_slug":"open-domain-question-answering","task_name":"Open-Domain Question Answering"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/open-domain-question-answering-on-natural-1","task":"Open-Domain Question Answering","dataset":"Natural Questions (short)","model":"EMDR2","rank_in_archive_order":1,"of":1,"metrics":{"Exact Match":"52.5"},"uses_additional_data":false},{"leaderboard":"/sota/open-domain-question-answering-on","task":"Open-Domain Question Answering","dataset":"WebQuestions","model":"EMDR2","rank_in_archive_order":4,"of":4,"metrics":{"Exact Match":"48.7"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-natural-questions","task":"Question Answering","dataset":"Natural Questions","model":"EMDR^2","rank_in_archive_order":9,"of":47,"metrics":{"EM":"52.5"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-triviaqa","task":"Question Answering","dataset":"TriviaQA","model":"EMDR2","rank_in_archive_order":29,"of":56,"metrics":{"EM":"71.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.05346","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.05346"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DevSinghSachan/art","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DevSinghSachan/emdr2","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_draft_wrong":2,"ran_fixture":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1},"community":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"928414780ee23e45","entry":"dot_product_scores","repo":"facebookresearch/dpr","repo_kind":"community","path":"dpr/models/biencoder.py","file_url":"https://github.com/facebookresearch/dpr/blob/HEAD/dpr/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"928414780ee23e45"}},{"code_sha256_prefix":"4c7d079ed8d8dd58","entry":"cosine_scores","repo":"facebookresearch/dpr","repo_kind":"community","path":"dpr/models/biencoder.py","file_url":"https://github.com/facebookresearch/dpr/blob/HEAD/dpr/models/biencoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4c7d079ed8d8dd58"}},{"code_sha256_prefix":"584d3ed8f5aa88ae","entry":"flatten","repo":"devsinghsachan/emdr2","repo_kind":"official","path":"megatron/model/emdr2_model.py","file_url":"https://github.com/devsinghsachan/emdr2/blob/HEAD/megatron/model/emdr2_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"584d3ed8f5aa88ae"}},{"code_sha256_prefix":"437fb284735a39a2","entry":"query_extended_context_t5_format","repo":"devsinghsachan/emdr2","repo_kind":"official","path":"megatron/model/emdr2_model.py","file_url":"https://github.com/devsinghsachan/emdr2/blob/HEAD/megatron/model/emdr2_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"437fb284735a39a2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}