{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automatic-and-universal-prompt-injection","title":"Automatic and Universal Prompt Injection Attacks against Large Language Models","arxiv_id":"2403.04957","date":"2024-03-07","proceeding":null,"authors":["Xiaogeng Liu","Zhiyuan Yu","Yizhe Zhang","Ning Zhang","Chaowei Xiao"],"abstract":"Large Language Models (LLMs) excel in processing and generating human language, powered by their ability to interpret and follow instructions. However, their capabilities can be exploited through prompt injection attacks. These attacks manipulate LLM-integrated applications into producing responses aligned with the attacker's injected content, deviating from the user's actual requests. The substantial risks posed by these attacks underscore the need for a thorough understanding of the threats. Yet, research in this area faces challenges due to the lack of a unified goal for such attacks and their reliance on manually crafted prompts, complicating comprehensive assessments of prompt injection robustness. We introduce a unified framework for understanding the objectives of prompt injection attacks and present an automated gradient-based method for generating highly effective and universal prompt injection data, even in the face of defensive measures. With only five training samples (0.3% relative to the test data), our attack can achieve superior performance compared with baselines. Our findings emphasize the importance of gradient-based testing, which can avoid overestimation of robustness, especially for defense mechanisms.","url_abs":"https://arxiv.org/abs/2403.04957v1","url_pdf":"https://arxiv.org/pdf/2403.04957v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automatic-and-universal-prompt-injection","repo_url":"https://github.com/sheltonliu-n/universal-prompt-injection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.04957","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.04957"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sheltonliu-n/universal-prompt-injection","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran":5,"unverified":3},"by_repo_kind":{"official":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7a1925d40005947b","entry":"check_for_attack_success","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"universal_prompt_injection.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/universal_prompt_injection.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a1925d40005947b"}},{"code_sha256_prefix":"c0332de7e8c09ed4","entry":"generate","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"universal_prompt_injection.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/universal_prompt_injection.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0332de7e8c09ed4"}},{"code_sha256_prefix":"b75ce23199955ac4","entry":"generate_pattern","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"universal_prompt_injection.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/universal_prompt_injection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b75ce23199955ac4"}},{"code_sha256_prefix":"5234b2286e5ed655","entry":"modify_sys_prompts","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"utils/string_utils.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/utils/string_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5234b2286e5ed655"}},{"code_sha256_prefix":"552d5602e36e866c","entry":"query_keyword","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"get_responses_universal.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/get_responses_universal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"552d5602e36e866c"}},{"code_sha256_prefix":"a505da2807081d09","entry":"query_target","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"check_answers.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/check_answers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a505da2807081d09"}},{"code_sha256_prefix":"e4d56bdd1dbd8b1c","entry":"query_target","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"utils/string_utils.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/utils/string_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4d56bdd1dbd8b1c"}},{"code_sha256_prefix":"688aaeba506e3329","entry":"get_embedding_matrix","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/utils/opt_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"688aaeba506e3329"}},{"code_sha256_prefix":"f556dda0785f3c77","entry":"get_embeddings","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/utils/opt_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f556dda0785f3c77"}},{"code_sha256_prefix":"76ca4b8b6f0c1fe9","entry":"token_gradients","repo":"sheltonliu-n/universal-prompt-injection","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/universal-prompt-injection/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76ca4b8b6f0c1fe9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}