{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-wolf-in-sheep-s-clothing-generalized-nested","title":"A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily","arxiv_id":"2311.08268","date":"2023-11-14","proceeding":null,"authors":["Peng Ding","Jun Kuang","Dan Ma","Xuezhi Cao","Yunsen Xian","Jiajun Chen","ShuJian Huang"],"abstract":"Large Language Models (LLMs), such as ChatGPT and GPT-4, are designed to provide useful and safe responses. However, adversarial prompts known as 'jailbreaks' can circumvent safeguards, leading LLMs to generate potentially harmful content. Exploring jailbreak prompts can help to better reveal the weaknesses of LLMs and further steer us to secure them. Unfortunately, existing jailbreak methods either suffer from intricate manual design or require optimization on other white-box models, which compromises either generalization or efficiency. In this paper, we generalize jailbreak prompt attacks into two aspects: (1) Prompt Rewriting and (2) Scenario Nesting. Based on this, we propose ReNeLLM, an automatic framework that leverages LLMs themselves to generate effective jailbreak prompts. Extensive experiments demonstrate that ReNeLLM significantly improves the attack success rate while greatly reducing the time cost compared to existing baselines. Our study also reveals the inadequacy of current defense methods in safeguarding LLMs. Finally, we analyze the failure of LLMs defense from the perspective of prompt execution priority, and propose corresponding defense strategies. We hope that our research can catalyze both the academic community and LLMs developers towards the provision of safer and more regulated LLMs. The code is available at https://github.com/NJUNLP/ReNeLLM.","url_abs":"https://arxiv.org/abs/2311.08268v4","url_pdf":"https://arxiv.org/pdf/2311.08268v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-wolf-in-sheep-s-clothing-generalized-nested","repo_url":"https://github.com/NJUNLP/ReNeLLM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2311.08268","atlas_url":"https://app.syntology.ai/?focus=2311.08268","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.08268"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NJUNLP/ReNeLLM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_violates":1,"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"76e1dbb6442681d1","entry":"chat_with_llama","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"renellm_tcps.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/renellm_tcps.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76e1dbb6442681d1"}},{"code_sha256_prefix":"3db7ff5e127560d0","entry":"data_reader","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3db7ff5e127560d0"}},{"code_sha256_prefix":"548b02bd8e478307","entry":"jailbroken_data_reader","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"548b02bd8e478307"}},{"code_sha256_prefix":"2a0c362d17ac1d21","entry":"mistral_responses","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"utils/llm_responses_utils.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/utils/llm_responses_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a0c362d17ac1d21"}},{"code_sha256_prefix":"14a84c2cbfebc413","entry":"precompute_freqs_cis","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"llama/llama/model.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/llama/llama/model.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14a84c2cbfebc413"}},{"code_sha256_prefix":"fe3bc07b02d43d0d","entry":"remove_number_prefix","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe3bc07b02d43d0d"}},{"code_sha256_prefix":"70bf6ebaafd266c4","entry":"reshape_for_broadcast","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"llama/llama/model.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/llama/llama/model.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70bf6ebaafd266c4"}},{"code_sha256_prefix":"7f04dbcdf72d68fd","entry":"apply_rotary_emb","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"llama/llama/model.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/llama/llama/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f04dbcdf72d68fd"}},{"code_sha256_prefix":"a25e322479780b3c","entry":"chatCompletion","repo":"NJUNLP/ReNeLLM","repo_kind":"official","path":"utils/llm_completion_utils.py","file_url":"https://github.com/NJUNLP/ReNeLLM/blob/HEAD/utils/llm_completion_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a25e322479780b3c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}