{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-adversarial-training-in-llms-with","title":"Efficient Adversarial Training in LLMs with Continuous Attacks","arxiv_id":"2405.15589","date":"2024-05-24","proceeding":null,"authors":["Sophie Xhonneux","Alessandro Sordoni","Stephan Günnemann","Gauthier Gidel","Leo Schwinn"],"abstract":"Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. In many domains, adversarial training has proven to be one of the most promising methods to reliably improve robustness against such attacks. Yet, in the context of LLMs, current methods for adversarial training are hindered by the high computational costs required to perform discrete adversarial attacks at each training iteration. We address this problem by instead calculating adversarial attacks in the continuous embedding space of the LLM, which is orders of magnitudes more efficient. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses: the first makes the model robust on continuous embedding attacks computed on an adversarial behaviour dataset; the second ensures the usefulness of the final model by fine-tuning on utility data. Moreover, we introduce C-AdvIPO, an adversarial variant of IPO that does not require utility data for adversarially robust alignment. Our empirical evaluation on five models from different families (Gemma, Phi3, Mistral, Zephyr, Llama2) and at different scales (2B, 3.8B, 7B) shows that both algorithms substantially enhance LLM robustness against discrete attacks (GCG, AutoDAN, PAIR), while maintaining utility. Our results demonstrate that robustness to continuous perturbations can extrapolate to discrete threat models. Thereby, we present a path toward scalable adversarial training algorithms for robustly aligning LLMs.","url_abs":"https://arxiv.org/abs/2405.15589v3","url_pdf":"https://arxiv.org/pdf/2405.15589v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-adversarial-training-in-llms-with","repo_url":"https://github.com/sophie-xhonneux/continuous-advtrain","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.15589","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.15589"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sophie-xhonneux/continuous-advtrain","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"unverified":3},"by_repo_kind":{"official":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"15be2e56e073a228","entry":"flatten_dict","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/database_handling.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/database_handling.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15be2e56e073a228"}},{"code_sha256_prefix":"780c913ff9f68d7b","entry":"get_chat_template","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/model_utils.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"780c913ff9f68d7b"}},{"code_sha256_prefix":"afa643eee62ab913","entry":"get_dataset_id","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/data.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afa643eee62ab913"}},{"code_sha256_prefix":"68d78c403eece7f7","entry":"get_dpo_loss","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/adversarial_training.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/adversarial_training.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68d78c403eece7f7"}},{"code_sha256_prefix":"711d5028382d4919","entry":"load_checkpoint","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/model_utils.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"711d5028382d4919"}},{"code_sha256_prefix":"e92c968a55357248","entry":"load_specific_dataset","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/data.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e92c968a55357248"}},{"code_sha256_prefix":"b5c7e326ac7ad2ed","entry":"log_1_minus_p_loss","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/adversarial_training.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/adversarial_training.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5c7e326ac7ad2ed"}},{"code_sha256_prefix":"dfef7ce9fc6e4f99","entry":"acquireLock","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/run_experiments.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/run_experiments.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfef7ce9fc6e4f99"}},{"code_sha256_prefix":"61ad493f36ea49b1","entry":"init_experiment","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/database_handling.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/database_handling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61ad493f36ea49b1"}},{"code_sha256_prefix":"06cd66993d331abf","entry":"load_model_and_tokenizer","repo":"sophie-xhonneux/continuous-advtrain","repo_kind":"official","path":"src/model_utils.py","file_url":"https://github.com/sophie-xhonneux/continuous-advtrain/blob/HEAD/src/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06cd66993d331abf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}