{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/poisonedrag-knowledge-poisoning-attacks-to","title":"PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models","arxiv_id":"2402.07867","date":"2024-02-12","proceeding":null,"authors":["Wei Zou","Runpeng Geng","Binghui Wang","Jinyuan Jia"],"abstract":"Large language models (LLMs) have achieved remarkable success due to their exceptional generative capabilities. Despite their success, they also have inherent limitations such as a lack of up-to-date knowledge and hallucination. Retrieval-Augmented Generation (RAG) is a state-of-the-art technique to mitigate these limitations. The key idea of RAG is to ground the answer generation of an LLM on external knowledge retrieved from a knowledge database. Existing studies mainly focus on improving the accuracy or efficiency of RAG, leaving its security largely unexplored. We aim to bridge the gap in this work. We find that the knowledge database in a RAG system introduces a new and practical attack surface. Based on this attack surface, we propose PoisonedRAG, the first knowledge corruption attack to RAG, where an attacker could inject a few malicious texts into the knowledge database of a RAG system to induce an LLM to generate an attacker-chosen target answer for an attacker-chosen target question. We formulate knowledge corruption attacks as an optimization problem, whose solution is a set of malicious texts. Depending on the background knowledge (e.g., black-box and white-box settings) of an attacker on a RAG system, we propose two solutions to solve the optimization problem, respectively. Our results show PoisonedRAG could achieve a 90% attack success rate when injecting five malicious texts for each target question into a knowledge database with millions of texts. We also evaluate several defenses and our results show they are insufficient to defend against PoisonedRAG, highlighting the need for new defenses.","url_abs":"https://arxiv.org/abs/2402.07867v3","url_pdf":"https://arxiv.org/pdf/2402.07867v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"poisonedrag-knowledge-poisoning-attacks-to","repo_url":"https://github.com/sleeepeer/poisonedrag","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"poisonedrag-knowledge-poisoning-attacks-to","repo_url":"https://github.com/agrzheng/GRADA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"answer-generation","task_name":"Answer Generation"},{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bart","method_name":"BART"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"rag","method_name":"RAG"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"set","method_name":"SET"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.07867","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.07867"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/agrzheng/GRADA","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sleeepeer/poisonedrag","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"ran_honours":1,"ran_fixture":1,"unverified":11},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1},"listed":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"27c0647865d4872e","entry":"ance_get_emb","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27c0647865d4872e"}},{"code_sha256_prefix":"a1619cb3b16730df","entry":"contriever_get_emb","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1619cb3b16730df"}},{"code_sha256_prefix":"a99ade31d3be8eec","entry":"dpr_get_emb","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a99ade31d3be8eec"}},{"code_sha256_prefix":"01971cc37e6fe583","entry":"gather_nograd","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/dist_utils.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/dist_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"01971cc37e6fe583"}},{"code_sha256_prefix":"309e71f4157403d6","entry":"hotflip_attack","repo":"agrzheng/GRADA","repo_kind":"listed","path":"src/attack.py","file_url":"https://github.com/agrzheng/GRADA/blob/HEAD/src/attack.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"309e71f4157403d6"}},{"code_sha256_prefix":"183fa5ce0ec5b32b","entry":"wrap_prompt","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/prompts.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"183fa5ce0ec5b32b"}},{"code_sha256_prefix":"7a4988952dd3a127","entry":"calculate_matches","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/evaluation.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a4988952dd3a127"}},{"code_sha256_prefix":"637b51b8df78832a","entry":"check_answer","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/evaluation.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"637b51b8df78832a"}},{"code_sha256_prefix":"9fb6e57668661c38","entry":"gather","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/dist_utils.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/dist_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fb6e57668661c38"}},{"code_sha256_prefix":"fa4ecbff4b9a354b","entry":"has_answer","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/evaluation.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fa4ecbff4b9a354b"}},{"code_sha256_prefix":"183aa13a4fe6a0cc","entry":"load_data","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/data.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"183aa13a4fe6a0cc"}},{"code_sha256_prefix":"0c21433d8fb86d42","entry":"load_dataset","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/data.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c21433d8fb86d42"}},{"code_sha256_prefix":"02909384e87fd820","entry":"load_retriever","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/contriever.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/contriever.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02909384e87fd820"}},{"code_sha256_prefix":"e238bddbf3216308","entry":"query_gpt","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"gen_adv.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/gen_adv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e238bddbf3216308"}},{"code_sha256_prefix":"e037474365501512","entry":"randomcrop","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/data.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e037474365501512"}},{"code_sha256_prefix":"b1ad8cd3a7168b7f","entry":"varsize_gather_nograd","repo":"sleeepeer/poisonedrag","repo_kind":"official","path":"src/contriever_src/dist_utils.py","file_url":"https://github.com/sleeepeer/poisonedrag/blob/HEAD/src/contriever_src/dist_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1ad8cd3a7168b7f"}},{"code_sha256_prefix":"96494dccb73d03f3","entry":"wrap_prompt","repo":"agrzheng/GRADA","repo_kind":"listed","path":"src/prompts.py","file_url":"https://github.com/agrzheng/GRADA/blob/HEAD/src/prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96494dccb73d03f3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}