{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/defending-large-language-models-against-1","title":"Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing","arxiv_id":"2402.16192","date":"2024-02-25","proceeding":null,"authors":["Jiabao Ji","Bairu Hou","Alexander Robey","George J. Pappas","Hamed Hassani","Yang Zhang","Eric Wong","Shiyu Chang"],"abstract":"Aligned large language models (LLMs) are vulnerable to jailbreaking attacks, which bypass the safeguards of targeted LLMs and fool them into generating objectionable content. While initial defenses show promise against token-based threat models, there do not exist defenses that provide robustness against semantic attacks and avoid unfavorable trade-offs between robustness and nominal performance. To meet this need, we propose SEMANTICSMOOTH, a smoothing-based defense that aggregates the predictions of multiple semantically transformed copies of a given input prompt. Experimental results demonstrate that SEMANTICSMOOTH achieves state-of-the-art robustness against GCG, PAIR, and AutoDAN attacks while maintaining strong nominal performance on instruction following benchmarks such as InstructionFollowing and AlpacaEval. The codes will be publicly available at https://github.com/UCSB-NLP-Chang/SemanticSmooth.","url_abs":"https://arxiv.org/abs/2402.16192v2","url_pdf":"https://arxiv.org/pdf/2402.16192v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"defending-large-language-models-against-1","repo_url":"https://github.com/ucsb-nlp-chang/semanticsmooth","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16192","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16192"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ucsb-nlp-chang/semanticsmooth","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b5252ac83a7c46c3","entry":"default_hparams","repo":"ucsb-nlp-chang/semanticsmooth","repo_kind":"official","path":"lib/hparams_registry.py","file_url":"https://github.com/ucsb-nlp-chang/semanticsmooth/blob/HEAD/lib/hparams_registry.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5252ac83a7c46c3"}},{"code_sha256_prefix":"77f69af078e09830","entry":"mean_pooling","repo":"ucsb-nlp-chang/semanticsmooth","repo_kind":"official","path":"lib/ensemble_policy.py","file_url":"https://github.com/ucsb-nlp-chang/semanticsmooth/blob/HEAD/lib/ensemble_policy.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77f69af078e09830"}},{"code_sha256_prefix":"4190042b24d3f36b","entry":"create_defense_from_config","repo":"ucsb-nlp-chang/semanticsmooth","repo_kind":"official","path":"lib/defenses.py","file_url":"https://github.com/ucsb-nlp-chang/semanticsmooth/blob/HEAD/lib/defenses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4190042b24d3f36b"}},{"code_sha256_prefix":"8cecc2a857c57545","entry":"create_ensemble_policy_model","repo":"ucsb-nlp-chang/semanticsmooth","repo_kind":"official","path":"lib/ensemble_policy.py","file_url":"https://github.com/ucsb-nlp-chang/semanticsmooth/blob/HEAD/lib/ensemble_policy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8cecc2a857c57545"}},{"code_sha256_prefix":"94699190b22cdf4c","entry":"create_perturbation","repo":"ucsb-nlp-chang/semanticsmooth","repo_kind":"official","path":"lib/perturbations.py","file_url":"https://github.com/ucsb-nlp-chang/semanticsmooth/blob/HEAD/lib/perturbations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94699190b22cdf4c"}},{"code_sha256_prefix":"b58775a1975c2aaa","entry":"merge_cmd_configs","repo":"ucsb-nlp-chang/semanticsmooth","repo_kind":"official","path":"lib/hparams_registry.py","file_url":"https://github.com/ucsb-nlp-chang/semanticsmooth/blob/HEAD/lib/hparams_registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b58775a1975c2aaa"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}