{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/don-t-say-no-jailbreaking-llm-by-suppressing","title":"Don't Say No: Jailbreaking LLM by Suppressing Refusal","arxiv_id":"2404.16369","date":"2024-04-25","proceeding":null,"authors":["Yukai Zhou","Zhijie Huang","Feiyang Lu","Zhan Qin","Wenjie Wang"],"abstract":"Ensuring the safety alignment of Large Language Models (LLMs) is crucial to generating responses consistent with human values. Despite their ability to recognize and avoid harmful queries, LLMs are vulnerable to jailbreaking attacks, where carefully crafted prompts seduce them to produce toxic content. One category of jailbreak attacks is reformulating the task as an optimization by eliciting the LLM to generate affirmative responses. However, such optimization objective has its own limitations, such as the restriction on the predefined objectionable behaviors, leading to suboptimal attack performance. In this study, we first uncover the reason why vanilla target loss is not optimal, then we explore and enhance the loss objective and introduce the DSN (Don't Say No) attack, which achieves successful attack by suppressing refusal. Another challenge in studying jailbreak attacks is the evaluation, as it is difficult to directly and accurately assess the harmfulness of the responses. The existing evaluation such as refusal keyword matching reveals numerous false positive and false negative instances. To overcome this challenge, we propose an Ensemble Evaluation pipeline that novelly incorporates Natural Language Inference (NLI) contradiction assessment and two external LLM evaluators. Extensive experiments demonstrate the potential of the DSN and effectiveness of Ensemble Evaluation compared to baseline methods.","url_abs":"https://arxiv.org/abs/2404.16369v2","url_pdf":"https://arxiv.org/pdf/2404.16369v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"don-t-say-no-jailbreaking-llm-by-suppressing","repo_url":"https://github.com/dsn-2024/dsn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"safety-alignment","task_name":"Safety Alignment"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.16369","atlas_url":"https://app.syntology.ai/?focus=2404.16369","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.16369"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dsn-2024/dsn","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"6d3ebbd46e153501","entry":"apply_cosine_decay","repo":"dsn-2024/dsn","repo_kind":"official","path":"llm_attacks/dsn/dsn_attack.py","file_url":"https://github.com/dsn-2024/dsn/blob/HEAD/llm_attacks/dsn/dsn_attack.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6d3ebbd46e153501"}},{"code_sha256_prefix":"74a7c644511d9142","entry":"from_harm_to_TF","repo":"dsn-2024/dsn","repo_kind":"official","path":"eval_ensemble_src/eval2_gpt.py","file_url":"https://github.com/dsn-2024/dsn/blob/HEAD/eval_ensemble_src/eval2_gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"74a7c644511d9142"}},{"code_sha256_prefix":"80bd78557542ffab","entry":"get_goals_and_targets","repo":"dsn-2024/dsn","repo_kind":"official","path":"eval_ensemble_src/get_everything.py","file_url":"https://github.com/dsn-2024/dsn/blob/HEAD/eval_ensemble_src/get_everything.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"80bd78557542ffab"}},{"code_sha256_prefix":"62b1b9e91747c5fc","entry":"get_model_name","repo":"dsn-2024/dsn","repo_kind":"official","path":"eval_ensemble_src/launch_majority_eval.py","file_url":"https://github.com/dsn-2024/dsn/blob/HEAD/eval_ensemble_src/launch_majority_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"62b1b9e91747c5fc"}},{"code_sha256_prefix":"bcb5d4c1f811371e","entry":"get_eval2_gpt4_results","repo":"dsn-2024/dsn","repo_kind":"official","path":"eval_ensemble_src/eval2_gpt.py","file_url":"https://github.com/dsn-2024/dsn/blob/HEAD/eval_ensemble_src/eval2_gpt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"bcb5d4c1f811371e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}