{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rear-a-relevance-aware-retrieval-augmented","title":"REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering","arxiv_id":"2402.17497","date":"2024-02-27","proceeding":null,"authors":["Yuhao Wang","Ruiyang Ren","Junyi Li","Wayne Xin Zhao","Jing Liu","Ji-Rong Wen"],"abstract":"Considering the limited internal parametric knowledge, retrieval-augmented generation (RAG) has been widely used to extend the knowledge scope of large language models (LLMs). Despite the extensive efforts on RAG research, in existing methods, LLMs cannot precisely assess the relevance of retrieved documents, thus likely leading to misleading or even incorrect utilization of external knowledge (eg., retrieved documents). To address this issue, in this paper, we propose REAR, a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA). As the key motivation, we aim to enhance the self-awareness regarding the reliability of external knowledge for LLMs, so as to adaptively utilize external knowledge in RAG systems. Specially, we develop a novel architecture for LLM-based RAG systems, by incorporating a specially designed assessment module that precisely assesses the relevance of retrieved documents. Furthermore, we propose an improved training method based on bi-granularity relevance fusion and noise-resistant training. By combining the improvements in both architecture and training, our proposed REAR can better utilize external knowledge by effectively perceiving the relevance of retrieved documents. Experiments on four open-domain QA tasks show that REAR significantly outperforms previous a number of competitive RAG approaches. Our codes can be accessed at https://github.com/RUCAIBox/REAR.","url_abs":"https://arxiv.org/abs/2402.17497v2","url_pdf":"https://arxiv.org/pdf/2402.17497v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rear-a-relevance-aware-retrieval-augmented","repo_url":"https://github.com/rucaibox/rear","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"open-domain-question-answering","task_name":"Open-Domain Question Answering"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bart","method_name":"BART"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"rag","method_name":"RAG"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2402.17497","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.17497"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rucaibox/rear","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/RUCAIBox/REAR","reach":{"status":"ok"}}],"summary":{"ran":5,"unverified":9},"by_repo_kind":{"official":{"samples":14,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"4e64f1e74615a17b","entry":"EM_compute","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/utils.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e64f1e74615a17b"}},{"code_sha256_prefix":"ab2b5996ca7899c5","entry":"F1_compute","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/utils.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab2b5996ca7899c5"}},{"code_sha256_prefix":"bb00e50823252c4e","entry":"flash_attn2_prepare_decoder_attention_mask","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/modeling_rearllama.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/modeling_rearllama.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bb00e50823252c4e"}},{"code_sha256_prefix":"cd9e184cbcee26b9","entry":"load_data","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/routing.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/routing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd9e184cbcee26b9"}},{"code_sha256_prefix":"fd045160cf301d81","entry":"process_ungrouped","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/dataset.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd045160cf301d81"}},{"code_sha256_prefix":"1fa4c96d77ce5f71","entry":"batch_generate","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/inf.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/inf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1fa4c96d77ce5f71"}},{"code_sha256_prefix":"6e2a58efe7401dca","entry":"batch_perplexity","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/inf.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/inf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e2a58efe7401dca"}},{"code_sha256_prefix":"da99009cb5e4b34f","entry":"consistency","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/routing.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/routing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da99009cb5e4b34f"}},{"code_sha256_prefix":"9f8e1820fdc2e76b","entry":"get_dataset","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/dataset.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f8e1820fdc2e76b"}},{"code_sha256_prefix":"73a6d0a707d6cc37","entry":"get_scores","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/inf.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/inf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73a6d0a707d6cc37"}},{"code_sha256_prefix":"3aa6f53b87553083","entry":"has_answer","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/utils.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3aa6f53b87553083"}},{"code_sha256_prefix":"f0ed5a58b6a31117","entry":"load_model_and_tokenizer","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/load_model.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/load_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0ed5a58b6a31117"}},{"code_sha256_prefix":"0b6a1efe994adc06","entry":"process_grouped","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/dataset.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b6a1efe994adc06"}},{"code_sha256_prefix":"acbbfc19df17afb0","entry":"reliability","repo":"RUCAIBox/REAR","repo_kind":"official","path":"rear/src/routing.py","file_url":"https://github.com/RUCAIBox/REAR/blob/HEAD/rear/src/routing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"acbbfc19df17afb0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}