{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-understanding-jailbreak-attacks-in","title":"Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis","arxiv_id":"2406.10794","date":"2024-06-16","proceeding":null,"authors":["Yuping Lin","Pengfei He","Han Xu","Yue Xing","Makoto Yamada","Hui Liu","Jiliang Tang"],"abstract":"Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. Although there are diverse jailbreak attack strategies, there is no unified understanding on why some methods succeed and others fail. This paper explores the behavior of harmful and harmless prompts in the LLM's representation space to investigate the intrinsic properties of successful jailbreak attacks. We hypothesize that successful attacks share some similar properties: They are effective in moving the representation of the harmful prompt towards the direction to the harmless prompts. We leverage hidden representations into the objective of existing jailbreak attacks to move the attacks along the acceptance direction, and conduct experiments to validate the above hypothesis using the proposed objective. We hope this study provides new insights into understanding how LLMs understand harmfulness information.","url_abs":"https://arxiv.org/abs/2406.10794v3","url_pdf":"https://arxiv.org/pdf/2406.10794v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-understanding-jailbreak-attacks-in","repo_url":"https://github.com/yuplin2333/representation-space-jailbreak","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.10794","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.10794"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuplin2333/representation-space-jailbreak","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":10},"by_repo_kind":{"official":{"samples":12,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"645d0aa0df490582","entry":"literal_eval","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"defense_paraphrase.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/defense_paraphrase.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"645d0aa0df490582"}},{"code_sha256_prefix":"57abbe64c30dbb2f","entry":"literal_eval","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"evaluate_llm.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/evaluate_llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57abbe64c30dbb2f"}},{"code_sha256_prefix":"162bb31349b73d4c","entry":"ask_judge_autodan","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"evaluate_llm.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/evaluate_llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"162bb31349b73d4c"}},{"code_sha256_prefix":"335a16bd75125cd0","entry":"ask_judge_pair","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"evaluate_llm.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/evaluate_llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"335a16bd75125cd0"}},{"code_sha256_prefix":"4e41cb9f01b69746","entry":"calculate_text_log_perplexity","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"defense_perplexity.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/defense_perplexity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e41cb9f01b69746"}},{"code_sha256_prefix":"fbd8844bf068105d","entry":"filter_candidates","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"jailbreak_gcg.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/jailbreak_gcg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbd8844bf068105d"}},{"code_sha256_prefix":"a8a775a24fb43efd","entry":"filter_candidates","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"jailbreak_ours_gcg.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/jailbreak_ours_gcg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8a775a24fb43efd"}},{"code_sha256_prefix":"ad88905d0561ced3","entry":"gpt_paraphrase","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"defense_paraphrase.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/defense_paraphrase.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad88905d0561ced3"}},{"code_sha256_prefix":"393e8574483752dd","entry":"perplexity_filter","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"defense_perplexity.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/defense_perplexity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"393e8574483752dd"}},{"code_sha256_prefix":"112080cc850b86b9","entry":"sample_control","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"jailbreak_gcg.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/jailbreak_gcg.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"112080cc850b86b9"}},{"code_sha256_prefix":"f8d61ebb2820cb0e","entry":"second_forward","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"jailbreak_gcg.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/jailbreak_gcg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8d61ebb2820cb0e"}},{"code_sha256_prefix":"699b1db1d82cab38","entry":"second_forward","repo":"yuplin2333/representation-space-jailbreak","repo_kind":"official","path":"jailbreak_ours_gcg.py","file_url":"https://github.com/yuplin2333/representation-space-jailbreak/blob/HEAD/jailbreak_ours_gcg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"699b1db1d82cab38"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}