{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2512-08892","title":"Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders","arxiv_id":"2512.08892","date":"2025-12-09","proceeding":"ICLR","authors":["Guangzhi Xiong","Zhenghao He","Bohan Liu","Sanchit Sinha","Aidong Zhang"],"abstract":"Retrieval-Augmented Generation (RAG) improves the factuality of large language models (LLMs) by grounding outputs in retrieved evidence, but faithfulness failures, where generations contradict or extend beyond the provided sources, remain a critical challenge. Existing hallucination detection methods for RAG often rely either on large-scale detector training, which requires substantial annotated data, or on querying external LLM judges, which leads to high inference costs. Although some approaches attempt to leverage internal representations of LLMs for hallucination detection, their accuracy remains limited. Motivated by recent advances in mechanistic interpretability, we employ sparse autoencoders (SAEs) to disentangle internal activations, successfully identifying features that are specifically triggered during RAG hallucinations. Building on a systematic pipeline of information-based feature selection and additive feature modeling, we introduce RAGLens, a lightweight hallucination detector that accurately flags unfaithful RAG outputs using LLM internal representations. RAGLens not only achieves superior detection performance compared to existing methods, but also provides interpretable rationales for its decisions, enabling effective post-hoc mitigation of unfaithful RAG. Finally, we justify our design choices and reveal new insights into the distribution of hallucination-related signals within LLMs. The code is available at https://github.com/Teddy-XiongGZ/RAGLens.","url_abs":"https://arxiv.org/abs/2512.08892","url_pdf":"https://arxiv.org/pdf/2512.08892","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2512.08892","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2512.08892"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Teddy-XiongGZ/RAGLens","reach":{"status":"ok"}}],"summary":{"unverified":11},"by_repo_kind":{"found_in_text":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"3fafa18bde72510f","entry":"chunk_and_tokenize","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sparsify/data.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sparsify/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fafa18bde72510f"}},{"code_sha256_prefix":"6f785f3051233faf","entry":"compute_mutual_information_chunked","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/utils.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f785f3051233faf"}},{"code_sha256_prefix":"a03ff408b1a15b33","entry":"compute_mutual_information_vectorized","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/utils.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a03ff408b1a15b33"}},{"code_sha256_prefix":"89cccd0c9373114f","entry":"encode_output_tokens","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sae_encoding.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sae_encoding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89cccd0c9373114f"}},{"code_sha256_prefix":"1fea12f6a78298c9","entry":"encode_outputs","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sae_encoding.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sae_encoding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1fea12f6a78298c9"}},{"code_sha256_prefix":"c839c072b645f3c8","entry":"fused_encoder","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sparsify/fused_encoder.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sparsify/fused_encoder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c839c072b645f3c8"}},{"code_sha256_prefix":"b44a86959a6f33aa","entry":"get_columns_all_equal","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sparsify/data.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sparsify/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b44a86959a6f33aa"}},{"code_sha256_prefix":"88eb5c5724c378e5","entry":"hf_generator","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/mitigation.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/mitigation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"88eb5c5724c378e5"}},{"code_sha256_prefix":"1930bbdc2f82dc62","entry":"quintic_newtonschulz","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sparsify/muon.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sparsify/muon.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1930bbdc2f82dc62"}},{"code_sha256_prefix":"251669896aaca2f6","entry":"sae_encoding","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/sae_encoding.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/sae_encoding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"251669896aaca2f6"}},{"code_sha256_prefix":"f867e8883cc4f7c1","entry":"vllm_generator","repo":"Teddy-XiongGZ/RAGLens","repo_kind":"found_in_text","path":"src/mitigation.py","file_url":"https://github.com/Teddy-XiongGZ/RAGLens/blob/HEAD/src/mitigation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f867e8883cc4f7c1"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}