{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-and-defending-against-indirect","title":"Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models","arxiv_id":"2312.14197","date":"2023-12-21","proceeding":null,"authors":["Jingwei Yi","Yueqi Xie","Bin Zhu","Emre Kiciman","Guangzhong Sun","Xing Xie","Fangzhao Wu"],"abstract":"The integration of large language models with external content has enabled applications such as Microsoft Copilot but also introduced vulnerabilities to indirect prompt injection attacks. In these attacks, malicious instructions embedded within external content can manipulate LLM outputs, causing deviations from user expectations. To address this critical yet under-explored issue, we introduce the first benchmark for indirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities. Using BIPIA, we evaluate existing LLMs and find them universally vulnerable. Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content. Based on these findings, we propose two novel defense mechanisms-boundary awareness and explicit reminder-to address these vulnerabilities in both black-box and white-box settings. Extensive experiments demonstrate that our black-box defense provides substantial mitigation, while our white-box defense reduces the attack success rate to near-zero levels, all while preserving the output quality of LLMs. We hope this work inspires further research into securing LLM applications and fostering their safe and reliable use.","url_abs":"https://arxiv.org/abs/2312.14197v4","url_pdf":"https://arxiv.org/pdf/2312.14197v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-and-defending-against-indirect","repo_url":"https://github.com/microsoft/BIPIA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[{"slug":"bipia","name":"BIPIA","full_name":"Benchmark of Indirect Prompt Injection Attacks"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.14197","atlas_url":"https://app.syntology.ai/?focus=2312.14197","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.14197"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/BIPIA","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"89f715e4644d2169","entry":"format_fn_factry","repo":"microsoft/BIPIA","repo_kind":"official","path":"bipia/metrics/regist.py","file_url":"https://github.com/microsoft/BIPIA/blob/HEAD/bipia/metrics/regist.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"89f715e4644d2169"}},{"code_sha256_prefix":"f632fd8875ef18c9","entry":"get_retry_time","repo":"microsoft/BIPIA","repo_kind":"official","path":"bipia/model/gpt.py","file_url":"https://github.com/microsoft/BIPIA/blob/HEAD/bipia/model/gpt.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f632fd8875ef18c9"}},{"code_sha256_prefix":"93f7f894f4a34cd8","entry":"depia_regist_fn","repo":"microsoft/BIPIA","repo_kind":"official","path":"bipia/metrics/regist.py","file_url":"https://github.com/microsoft/BIPIA/blob/HEAD/bipia/metrics/regist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"93f7f894f4a34cd8"}},{"code_sha256_prefix":"261a76dfd405bb0e","entry":"task_irrelevant_regist_fn","repo":"microsoft/BIPIA","repo_kind":"official","path":"bipia/metrics/regist.py","file_url":"https://github.com/microsoft/BIPIA/blob/HEAD/bipia/metrics/regist.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"261a76dfd405bb0e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}