{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gradient-cuff-detecting-jailbreak-attacks-on","title":"Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes","arxiv_id":"2403.00867","date":"2024-03-01","proceeding":null,"authors":["Xiaomeng Hu","Pin-Yu Chen","Tsung-Yi Ho"],"abstract":"Large Language Models (LLMs) are becoming a prominent generative AI tool, where the user enters a query and the LLM generates an answer. To reduce harm and misuse, efforts have been made to align these LLMs to human values using advanced training techniques such as Reinforcement Learning from Human Feedback (RLHF). However, recent studies have highlighted the vulnerability of LLMs to adversarial jailbreak attempts aiming at subverting the embedded safety guardrails. To address this challenge, this paper defines and investigates the Refusal Loss of LLMs and then proposes a method called Gradient Cuff to detect jailbreak attempts. Gradient Cuff exploits the unique properties observed in the refusal loss landscape, including functional values and its smoothness, to design an effective two-step detection strategy. Experimental results on two aligned LLMs (LLaMA-2-7B-Chat and Vicuna-7B-V1.5) and six types of jailbreak attacks (GCG, AutoDAN, PAIR, TAP, Base64, and LRL) show that Gradient Cuff can significantly improve the LLM's rejection capability for malicious jailbreak queries, while maintaining the model's performance for benign user queries by adjusting the detection threshold.","url_abs":"https://arxiv.org/abs/2403.00867v3","url_pdf":"https://arxiv.org/pdf/2403.00867v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.00867","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.00867"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/SheltonLiu-N/AutoDAN","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/llm-attacks/llm-attacks","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/patrickrchao/JailbreakingLLMs","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/RICommunity/TAP","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_draft_wrong":1,"ran_fixture":1,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":11,"ran":9,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"95b0058c5ba905d2","entry":"GCGJudge","repo":"patrickrchao/JailbreakingLLMs","repo_kind":"found_in_text","path":"judges.py","file_url":"https://github.com/patrickrchao/JailbreakingLLMs/blob/HEAD/judges.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95b0058c5ba905d2"}},{"code_sha256_prefix":"f5838bba583ba02c","entry":"JudgeBase","repo":"patrickrchao/JailbreakingLLMs","repo_kind":"found_in_text","path":"judges.py","file_url":"https://github.com/patrickrchao/JailbreakingLLMs/blob/HEAD/judges.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f5838bba583ba02c"}},{"code_sha256_prefix":"7337ffebcca7ec8c","entry":"extract_json","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"common.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/common.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7337ffebcca7ec8c"}},{"code_sha256_prefix":"4d558fe30721ce34","entry":"get_attacker_system_prompt","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"system_prompts.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/system_prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d558fe30721ce34"}},{"code_sha256_prefix":"9b383d07783e38a8","entry":"get_evaluator_system_prompt_for_judge","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"system_prompts.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/system_prompts.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b383d07783e38a8"}},{"code_sha256_prefix":"57bae653b9458268","entry":"get_evaluator_system_prompt_for_on_topic","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"system_prompts.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/system_prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57bae653b9458268"}},{"code_sha256_prefix":"e5d6b48c8179a8d3","entry":"get_init_msg","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"common.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5d6b48c8179a8d3"}},{"code_sha256_prefix":"60e7444ebd37a613","entry":"get_judge_system_prompt","repo":"patrickrchao/JailbreakingLLMs","repo_kind":"found_in_text","path":"judges.py","file_url":"https://github.com/patrickrchao/JailbreakingLLMs/blob/HEAD/judges.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60e7444ebd37a613"}},{"code_sha256_prefix":"27665c6bc88f1a85","entry":"prune","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"main_TAP.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/main_TAP.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27665c6bc88f1a85"}},{"code_sha256_prefix":"8b0c27ec8691436a","entry":"clean_attacks_and_convs","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"main_TAP.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/main_TAP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b0c27ec8691436a"}},{"code_sha256_prefix":"8b16e6a7085993f1","entry":"random_string","repo":"RICommunity/TAP","repo_kind":"found_in_text","path":"common.py","file_url":"https://github.com/RICommunity/TAP/blob/HEAD/common.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b16e6a7085993f1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}