{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/metagen-blended-rag-higher-accuracy-for","title":"MetaGen Blended RAG: Higher Accuracy for Domain-Specific Q&A Without Fine-Tuning","arxiv_id":"2505.18247","date":"2025-05-23","proceeding":null,"authors":["Kunal Sawarkar","Shivam R. Solanki","Abhilasha Mangal"],"abstract":"Despite the widespread exploration of Retrieval-Augmented Generation (RAG), its deployment in enterprises for domain-specific datasets remains limited due to poor answer accuracy. These corpora, often shielded behind firewalls in private enterprise knowledge bases, having complex, domain-specific terminology, rarely seen by LLMs during pre-training; exhibit significant semantic variability across domains (like networking, military, or legal, etc.), or even within a single domain like medicine, and thus result in poor context precision for RAG systems. Currently, in such situations, fine-tuning or RAG with fine-tuning is attempted, but these approaches are slow, expensive, and lack generalization for accuracy as the new domain-specific data emerges. We propose an approach for Enterprise Search that focuses on enhancing the retriever for a domain-specific corpus through hybrid query indexes and metadata enrichment. This 'MetaGen Blended RAG' method constructs a metadata generation pipeline using key concepts, topics, and acronyms, and then creates a metadata-enriched hybrid index with boosted search queries. This approach avoids overfitting and generalizes effectively across domains. On the PubMedQA benchmark for the biomedical domain, the proposed method achieves 82% retrieval accuracy and 77% RAG accuracy, surpassing all previous RAG accuracy results without fine-tuning and sets a new benchmark for zero-shot results while outperforming much larger models like GPT3.5. The results are even comparable to the best fine-tuned models on this dataset, and we further demonstrate the robustness and scalability of the approach by evaluating it on other Q&A datasets like SQuAD, NQ etc.","url_abs":"https://arxiv.org/abs/2505.18247v1","url_pdf":"https://arxiv.org/pdf/2505.18247v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"metagen-blended-rag-higher-accuracy-for","repo_url":"https://github.com/ibm-self-serve-assets/metagen-blended-rag","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[{"method_slug":"rag","method_name":"RAG"}],"datasets_introduced":[{"slug":"pubmedqa-corpus-with-metadata","name":"PubMedQA corpus with metadata","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/few-shot-learning-on-pubmedqa","task":"Few-Shot Learning","dataset":"PubMedQA","model":"MetaGen Blended RAG (zero-shot)","rank_in_archive_order":1,"of":2,"metrics":{"Accuracy":"77.9"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-pubmedqa","task":"Question Answering","dataset":"PubMedQA","model":"MetaGen Blended RAG (zero-shot)","rank_in_archive_order":9,"of":30,"metrics":{"Accuracy":"77.9"},"uses_additional_data":false},{"leaderboard":"/sota/rag-on-pubmedqa-corpus-with-metadata","task":"RAG","dataset":"PubMedQA corpus with metadata","model":"MetaGen Blended RAG","rank_in_archive_order":1,"of":1,"metrics":{"ANS-EM":"77.90"},"uses_additional_data":false},{"leaderboard":"/sota/retrieval-on-pubmedqa","task":"Retrieval","dataset":"PubMedQA","model":"MetaGen Blended RAG","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy (Top-1)":"82.1"},"uses_additional_data":false},{"leaderboard":"/sota/retrieval-on-pubmedqa-corpus-with-metadata","task":"Retrieval","dataset":"PubMedQA corpus with metadata","model":"MetaGen Blended RAG","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy (Top-1)":"82.1"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}