{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/robust-retrieval-augmented-generation-for","title":"Robust Retrieval Augmented Generation for Zero-shot Slot Filling","arxiv_id":"2108.13934","date":"2021-08-31","proceeding":"EMNLP 2021 11","authors":["Michael Glass","Gaetano Rossiello","Md Faisal Mahbub Chowdhury","Alfio Gliozzo"],"abstract":"Automatically inducing high quality knowledge graphs from a given collection of documents still remains a challenging problem in AI. One way to make headway for this problem is through advancements in a related task known as slot filling. In this task, given an entity query in form of [Entity, Slot, ?], a system is asked to fill the slot by generating or extracting the missing value exploiting evidence extracted from relevant passage(s) in the given document collection. The recent works in the field try to solve this task in an end-to-end fashion using retrieval-based language models. In this paper, we present a novel approach to zero-shot slot filling that extends dense passage retrieval with hard negatives and robust training procedures for retrieval augmented generation models. Our model reports large improvements on both T-REx and zsRE slot filling datasets, improving both passage retrieval and slot value generation, and ranking at the top-1 position in the KILT leaderboard. Moreover, we demonstrate the robustness of our system showing its domain adaptation capability on a new variant of the TACRED dataset for slot filling, through a combination of zero/few-shot learning. We release the source code and pre-trained models.","url_abs":"https://arxiv.org/abs/2108.13934v2","url_pdf":"https://arxiv.org/pdf/2108.13934v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"robust-retrieval-augmented-generation-for","repo_url":"https://github.com/ibm/kgi-slot-filling","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"robust-retrieval-augmented-generation-for","repo_url":"https://github.com/IBM/retrieve-write-slot-filling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"passage-retrieval","task_name":"Passage Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"},{"task_slug":"slot-filling","task_name":"Slot Filling"},{"task_slug":"zero-shot-slot-filling","task_name":"Zero-shot Slot Filling"},{"task_slug":"slot-filling-1","task_name":"slot-filling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/zero-shot-slot-filling-on-t-rex","task":"Zero-shot Slot Filling","dataset":"T-REx","model":"DPRDNS+RAG","rank_in_archive_order":1,"of":6,"metrics":{"R-Prec":"74.34","R@5":"82.89"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-t-rex","task":"Zero-shot Slot Filling","dataset":"T-REx","model":"DPRBM25+RAG","rank_in_archive_order":2,"of":6,"metrics":{"R-Prec":"65.02","R@5":"75.52"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-t-rex","task":"Zero-shot Slot Filling","dataset":"T-REx","model":"DPRNQ+RAG","rank_in_archive_order":3,"of":6,"metrics":{"R-Prec":"53.04","R@5":"65.54"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-t-rex","task":"Zero-shot Slot Filling","dataset":"T-REx","model":"DPRBM25","rank_in_archive_order":4,"of":6,"metrics":{"R-Prec":"49.02","R@5":"63.34"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-t-rex","task":"Zero-shot Slot Filling","dataset":"T-REx","model":"DPRDNS","rank_in_archive_order":5,"of":6,"metrics":{"R-Prec":"42.62","R@5":"55.09"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-t-rex","task":"Zero-shot Slot Filling","dataset":"T-REx","model":"DPRNQ","rank_in_archive_order":6,"of":6,"metrics":{"R-Prec":"19.5","R@5":"29.8"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-zsre","task":"Zero-shot Slot Filling","dataset":"zsRE","model":"DPRDNS+RAG","rank_in_archive_order":1,"of":6,"metrics":{"R-Prec":"98.60","R@5":"99.70"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-zsre","task":"Zero-shot Slot Filling","dataset":"zsRE","model":"DPRDNS","rank_in_archive_order":2,"of":6,"metrics":{"R-Prec":"97.53","R@5":"99.3"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-zsre","task":"Zero-shot Slot Filling","dataset":"zsRE","model":"DPRBM25+RAG","rank_in_archive_order":3,"of":6,"metrics":{"R-Prec":"96.89","R@5":"98.01"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-zsre","task":"Zero-shot Slot Filling","dataset":"zsRE","model":"DPRBM25","rank_in_archive_order":4,"of":6,"metrics":{"R-Prec":"94.55","R@5":"98.17"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-zsre","task":"Zero-shot Slot Filling","dataset":"zsRE","model":"DPRNQ+RAG","rank_in_archive_order":5,"of":6,"metrics":{"R-Prec":"68.13","R@5":"79.19"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-slot-filling-on-zsre","task":"Zero-shot Slot Filling","dataset":"zsRE","model":"DPRNQ","rank_in_archive_order":6,"of":6,"metrics":{"R-Prec":"45.49","R@5":"60.77"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.13934","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.13934"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ibm/kgi-slot-filling","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/retrieve-write-slot-filling","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"listed":{"samples":8,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e7e75981cb464788","entry":"normalize_answer","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/rag_util.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/rag_util.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7e75981cb464788"}},{"code_sha256_prefix":"d890771bd3e4b3f7","entry":"clean_text","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/kilt_passage_corpus.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/kilt_passage_corpus.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d890771bd3e4b3f7"}},{"code_sha256_prefix":"50ec835b3753c6ec","entry":"find_para_range","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/kilt_passage_corpus.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/kilt_passage_corpus.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50ec835b3753c6ec"}},{"code_sha256_prefix":"e57ce1467b8c1988","entry":"get_relation_from_inst","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/dataset_stats.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/dataset_stats.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e57ce1467b8c1988"}},{"code_sha256_prefix":"0fa853d294de8b23","entry":"matching_passage_ids","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"dpr/dpr_kilt_slot_filling_dataset.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/dpr/dpr_kilt_slot_filling_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0fa853d294de8b23"}},{"code_sha256_prefix":"89afe8d9e930bb10","entry":"postprocess_docs","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/rag_util.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/rag_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"89afe8d9e930bb10"}},{"code_sha256_prefix":"1d4a7d4d6e3eb7e0","entry":"prefix_allowed_tokens_fn","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/rag_client_server_apply.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/rag_client_server_apply.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d4a7d4d6e3eb7e0"}},{"code_sha256_prefix":"f92394bf8b4685cc","entry":"prepare_seq2seq_batch","repo":"IBM/retrieve-write-slot-filling","repo_kind":"listed","path":"slot_filling/rag_util.py","file_url":"https://github.com/IBM/retrieve-write-slot-filling/blob/HEAD/slot_filling/rag_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f92394bf8b4685cc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}