{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unveiling-the-safety-of-gpt-4o-an-empirical","title":"Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks","arxiv_id":"2406.06302","date":"2024-06-10","proceeding":null,"authors":["Zonghao Ying","Aishan Liu","Xianglong Liu","DaCheng Tao"],"abstract":"The recent release of GPT-4o has garnered widespread attention due to its powerful general capabilities. While its impressive performance is widely acknowledged, its safety aspects have not been sufficiently explored. Given the potential societal impact of risky content generated by advanced generative AI such as GPT-4o, it is crucial to rigorously evaluate its safety. In response to this question, this paper for the first time conducts a rigorous evaluation of GPT-4o against jailbreak attacks. Specifically, this paper adopts a series of multi-modal and uni-modal jailbreak attacks on 4 commonly used benchmarks encompassing three modalities (ie, text, speech, and image), which involves the optimization of over 4,000 initial text queries and the analysis and statistical evaluation of nearly 8,000+ response on GPT-4o. Our extensive experiments reveal several novel observations: (1) In contrast to the previous version (such as GPT-4V), GPT-4o has enhanced safety in the context of text modality jailbreak; (2) The newly introduced audio modality opens up new attack vectors for jailbreak attacks on GPT-4o; (3) Existing black-box multimodal jailbreak attack methods are largely ineffective against GPT-4o and GPT-4V. These findings provide critical insights into the safety implications of GPT-4o and underscore the need for robust alignment guardrails in large models. Our code is available at \\url{https://github.com/NY1024/Jailbreak_GPT4o}.","url_abs":"https://arxiv.org/abs/2406.06302v2","url_pdf":"https://arxiv.org/pdf/2406.06302v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unveiling-the-safety-of-gpt-4o-an-empirical","repo_url":"https://github.com/ny1024/jailbreak_gpt4o","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2406.06302","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.06302"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ny1024/jailbreak_gpt4o","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"1f37e736f5e4836d","entry":"check_string","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/J1.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/J1.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f37e736f5e4836d"}},{"code_sha256_prefix":"cdf53979a6cb611c","entry":"function_a","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"AdvBench/category.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/AdvBench/category.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cdf53979a6cb611c"}},{"code_sha256_prefix":"113482536c0c82ac","entry":"gpt35","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/J3.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/J3.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"113482536c0c82ac"}},{"code_sha256_prefix":"3439800ee446e2ac","entry":"gpt35","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/J4.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/J4.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3439800ee446e2ac"}},{"code_sha256_prefix":"3fc046cfb80b3004","entry":"gpt35","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/Liufina.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/Liufina.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fc046cfb80b3004"}},{"code_sha256_prefix":"17e0717961098449","entry":"gpt35","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/Liugov.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/Liugov.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17e0717961098449"}},{"code_sha256_prefix":"c9b78b4bbba027c2","entry":"gpt35","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/Liuhealth.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/Liuhealth.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9b78b4bbba027c2"}},{"code_sha256_prefix":"2b98eca89791042c","entry":"gpt35","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"Judgement&Auxiliary/Liulegal.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/Judgement%26Auxiliary/Liulegal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b98eca89791042c"}},{"code_sha256_prefix":"8c420db9256b7514","entry":"gpt4o","repo":"ny1024/jailbreak_gpt4o","repo_kind":"official","path":"AdvBench/category.py","file_url":"https://github.com/ny1024/jailbreak_gpt4o/blob/HEAD/AdvBench/category.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8c420db9256b7514"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}