{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rare-retrieval-augmented-reasoning-modeling","title":"RARE: Retrieval-Augmented Reasoning Modeling","arxiv_id":"2503.23513","date":"2025-03-30","proceeding":null,"authors":["Zhengren Wang","Jiayang Yu","Dongsheng Ma","Zhe Chen","Yu Wang","Zhiyu Li","Feiyu Xiong","Yanfeng Wang","Weinan E","Linpeng Tang","Wentao Zhang"],"abstract":"Domain-specific intelligence demands specialized knowledge and sophisticated reasoning for problem-solving, posing significant challenges for large language models (LLMs) that struggle with knowledge hallucination and inadequate reasoning capabilities under constrained parameter budgets. Inspired by Bloom's Taxonomy in educational theory, we propose Retrieval-Augmented Reasoning Modeling (RARE), a novel paradigm that decouples knowledge storage from reasoning optimization. RARE externalizes domain knowledge to retrievable sources and internalizes domain-specific reasoning patterns during training. Specifically, by injecting retrieved knowledge into training prompts with masked losses, RARE transforms learning objectives from rote memorization to contextualized reasoning. It enables models to bypass parameter-intensive memorization and prioritize the development of higher-order cognitive processes. Extensive experiments demonstrate that lightweight RARE-trained models (e.g., Llama-3.1-8B) could achieve state-of-the-art performance, surpassing retrieval-augmented GPT-4 and DeepSeek-R1 up to approximately 20\\% accuracy. RARE establishes a paradigm shift where maintainable external knowledge bases synergize with compact, reasoning-optimized models, collectively driving more scalable domain-specific intelligence.","url_abs":"https://arxiv.org/abs/2503.23513v2","url_pdf":"https://arxiv.org/pdf/2503.23513v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rare-retrieval-augmented-reasoning-modeling","repo_url":"https://github.com/open-dataflow/rare","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.23513","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.23513"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/open-dataflow/rare","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":14},"by_repo_kind":{"official":{"samples":14,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8b213cd7535cb82d","entry":"check_input_file","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/api_infer_post.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/api_infer_post.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b213cd7535cb82d"}},{"code_sha256_prefix":"31eca3c5db5632d2","entry":"extract_option","repo":"open-dataflow/rare","repo_kind":"official","path":"RL_KTO/process_kto.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/RL_KTO/process_kto.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31eca3c5db5632d2"}},{"code_sha256_prefix":"43b4dce1bb401779","entry":"format_prompt","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/vllm_infer_mm.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/vllm_infer_mm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"43b4dce1bb401779"}},{"code_sha256_prefix":"0fb897a1530a0ca3","entry":"format_prompt","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/vllm_infer_text.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/vllm_infer_text.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0fb897a1530a0ca3"}},{"code_sha256_prefix":"7734148d35457b70","entry":"get_stop_tokens","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/vllm_infer_mm.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/vllm_infer_mm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7734148d35457b70"}},{"code_sha256_prefix":"a838fd7b11bb9013","entry":"get_stop_tokens","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/vllm_infer_text.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/vllm_infer_text.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a838fd7b11bb9013"}},{"code_sha256_prefix":"fc64cae458054855","entry":"load_data","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/api_infer_post.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/api_infer_post.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc64cae458054855"}},{"code_sha256_prefix":"0cb12974f022de15","entry":"load_dataset","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/vllm_infer_mm.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/vllm_infer_mm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0cb12974f022de15"}},{"code_sha256_prefix":"5574a27348d23871","entry":"load_dataset","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/vllm_infer_text.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/vllm_infer_text.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5574a27348d23871"}},{"code_sha256_prefix":"0ca1b69ca52ee35c","entry":"load_dataset","repo":"open-dataflow/rare","repo_kind":"official","path":"process/process_medqa.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/process/process_medqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ca1b69ca52ee35c"}},{"code_sha256_prefix":"eef3708cdaa2c7e6","entry":"prepare_queue","repo":"open-dataflow/rare","repo_kind":"official","path":"inference/api_infer_post.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/inference/api_infer_post.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eef3708cdaa2c7e6"}},{"code_sha256_prefix":"bf7b0dc382946a22","entry":"process_dataset","repo":"open-dataflow/rare","repo_kind":"official","path":"process/process_medqa.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/process/process_medqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bf7b0dc382946a22"}},{"code_sha256_prefix":"5d7f6062b1de0214","entry":"process_item","repo":"open-dataflow/rare","repo_kind":"official","path":"process/process_pubhealth.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/process/process_pubhealth.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d7f6062b1de0214"}},{"code_sha256_prefix":"271b7030a5cdfb52","entry":"process_line","repo":"open-dataflow/rare","repo_kind":"official","path":"process/process_mmrait.py","file_url":"https://github.com/open-dataflow/rare/blob/HEAD/process/process_mmrait.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"271b7030a5cdfb52"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}