{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/through-the-stealth-lens-rethinking-attacks","title":"Through the Stealth Lens: Rethinking Attacks and Defenses in RAG","arxiv_id":"2506.04390","date":"2025-06-04","proceeding":null,"authors":["Sarthak Choudhary","Nils Palumbo","Ashish Hooda","Krishnamurthy Dj Dvijotham","Somesh Jha"],"abstract":"Retrieval-augmented generation (RAG) systems are vulnerable to attacks that inject poisoned passages into the retrieved set, even at low corruption rates. We show that existing attacks are not designed to be stealthy, allowing reliable detection and mitigation. We formalize stealth using a distinguishability-based security game. If a few poisoned passages are designed to control the response, they must differentiate themselves from benign ones, inherently compromising stealth. This motivates the need for attackers to rigorously analyze intermediate signals involved in generation$\\unicode{x2014}$such as attention patterns or next-token probability distributions$\\unicode{x2014}$to avoid easily detectable traces of manipulation. Leveraging attention patterns, we propose a passage-level score$\\unicode{x2014}$the Normalized Passage Attention Score$\\unicode{x2014}$used by our Attention-Variance Filter algorithm to identify and filter potentially poisoned passages. This method mitigates existing attacks, improving accuracy by up to $\\sim 20 \\%$ over baseline defenses. To probe the limits of attention-based defenses, we craft stealthier adaptive attacks that obscure such traces, achieving up to $35 \\%$ attack success rate, and highlight the challenges in improving stealth.","url_abs":"https://arxiv.org/abs/2506.04390v1","url_pdf":"https://arxiv.org/pdf/2506.04390v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"through-the-stealth-lens-rethinking-attacks","repo_url":"https://github.com/sarthak-choudhary/stealthy_attacks_against_rag","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.04390","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.04390"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sarthak-choudhary/stealthy_attacks_against_rag","reach":null}],"summary":{"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"568ec8fecf60846b","entry":"filter_by_attention_score","repo":"sarthak-choudhary/stealthy_attacks_against_rag","repo_kind":"official","path":"src/filtering.py","file_url":"https://github.com/sarthak-choudhary/stealthy_attacks_against_rag/blob/HEAD/src/filtering.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"568ec8fecf60846b"}},{"code_sha256_prefix":"4f9e005153e0ef06","entry":"wrap_prompt","repo":"sarthak-choudhary/stealthy_attacks_against_rag","repo_kind":"official","path":"src/filtering.py","file_url":"https://github.com/sarthak-choudhary/stealthy_attacks_against_rag/blob/HEAD/src/filtering.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f9e005153e0ef06"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}