{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autodan-generating-stealthy-jailbreak-prompts","title":"AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models","arxiv_id":"2310.04451","date":"2023-10-03","proceeding":null,"authors":["Xiaogeng Liu","Nan Xu","Muhao Chen","Chaowei Xiao"],"abstract":"The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with human feedback. However, these large models remain susceptible to jailbreak attacks, where adversaries manipulate prompts to elicit malicious outputs that should not be given by aligned LLMs. Investigating jailbreak prompts can lead us to delve into the limitations of LLMs and further guide us to secure them. Unfortunately, existing jailbreak techniques suffer from either (1) scalability issues, where attacks heavily rely on manual crafting of prompts, or (2) stealthiness problems, as attacks depend on token-based algorithms to generate prompts that are often semantically meaningless, making them susceptible to detection through basic perplexity testing. In light of these challenges, we intend to answer this question: Can we develop an approach that can automatically generate stealthy jailbreak prompts? In this paper, we introduce AutoDAN, a novel jailbreak attack against aligned LLMs. AutoDAN can automatically generate stealthy jailbreak prompts by the carefully designed hierarchical genetic algorithm. Extensive evaluations demonstrate that AutoDAN not only automates the process while preserving semantic meaningfulness, but also demonstrates superior attack strength in cross-model transferability, and cross-sample universality compared with the baseline. Moreover, we also compare AutoDAN with perplexity-based defense methods and show that AutoDAN can bypass them effectively.","url_abs":"https://arxiv.org/abs/2310.04451v2","url_pdf":"https://arxiv.org/pdf/2310.04451v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autodan-generating-stealthy-jailbreak-prompts","repo_url":"https://github.com/sheltonliu-n/autodan","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"autodan-generating-stealthy-jailbreak-prompts","repo_url":"https://github.com/NJUNLP/ReNeLLM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.04451","atlas_url":"https://app.syntology.ai/?focus=2310.04451","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.04451"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/SheltonLiu-N/AutoDAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NJUNLP/ReNeLLM","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sheltonliu-n/autodan","reach":null}],"summary":{"ran_fixture":2,"ran":1,"ran_draft_wrong":3,"unverified":4},"by_repo_kind":{"official":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7c9e974e56708d13","entry":"apply_crossover_and_mutation","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c9e974e56708d13"}},{"code_sha256_prefix":"9b962dc06655770e","entry":"apply_gpt_mutation","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b962dc06655770e"}},{"code_sha256_prefix":"7a1925d40005947b","entry":"check_for_attack_success","repo":"SheltonLiu-N/AutoDAN","repo_kind":"official","path":"autodan_ga_eval.py","file_url":"https://github.com/SheltonLiu-N/AutoDAN/blob/HEAD/autodan_ga_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a1925d40005947b"}},{"code_sha256_prefix":"f7269d0d60790e7b","entry":"crossover","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7269d0d60790e7b"}},{"code_sha256_prefix":"c0332de7e8c09ed4","entry":"generate","repo":"SheltonLiu-N/AutoDAN","repo_kind":"official","path":"autodan_ga_eval.py","file_url":"https://github.com/SheltonLiu-N/AutoDAN/blob/HEAD/autodan_ga_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0332de7e8c09ed4"}},{"code_sha256_prefix":"1fe7dbe6f5cc4593","entry":"roulette_wheel_selection","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1fe7dbe6f5cc4593"}},{"code_sha256_prefix":"830a85ed7000f19b","entry":"autodan_sample_control","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"830a85ed7000f19b"}},{"code_sha256_prefix":"272207fb8d21ce30","entry":"get_developer","repo":"SheltonLiu-N/AutoDAN","repo_kind":"official","path":"autodan_ga_eval.py","file_url":"https://github.com/SheltonLiu-N/AutoDAN/blob/HEAD/autodan_ga_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"272207fb8d21ce30"}},{"code_sha256_prefix":"b4c5d45b78d6c1b9","entry":"gpt_mutate","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4c5d45b78d6c1b9"}},{"code_sha256_prefix":"d013d2ff9040eecb","entry":"replace_with_synonyms","repo":"sheltonliu-n/autodan","repo_kind":"official","path":"utils/opt_utils.py","file_url":"https://github.com/sheltonliu-n/autodan/blob/HEAD/utils/opt_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d013d2ff9040eecb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}