{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/virus-harmful-fine-tuning-attack-for-large","title":"Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation","arxiv_id":"2501.17433","date":"2025-01-29","proceeding":null,"authors":["Tiansheng Huang","Sihao Hu","Fatih Ilhan","Selim Furkan Tekin","Ling Liu"],"abstract":"Recent research shows that Large Language Models (LLMs) are vulnerable to harmful fine-tuning attacks -- models lose their safety alignment ability after fine-tuning on a few harmful samples. For risk mitigation, a guardrail is typically used to filter out harmful samples before fine-tuning. By designing a new red-teaming method, we in this paper show that purely relying on the moderation guardrail for data filtration is not reliable. Our proposed attack method, dubbed Virus, easily bypasses the guardrail moderation by slightly modifying the harmful data. Experimental results show that the harmful data optimized by Virus is not detectable by the guardrail with up to 100\\% leakage ratio, and can simultaneously achieve superior attack performance. Finally, the key message we want to convey through this paper is that: \\textbf{it is reckless to consider guardrail moderation as a clutch at straws towards harmful fine-tuning attack}, as it cannot solve the inherent safety issue of the pre-trained LLMs. Our code is available at https://github.com/git-disl/Virus","url_abs":"https://arxiv.org/abs/2501.17433v1","url_pdf":"https://arxiv.org/pdf/2501.17433v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"virus-harmful-fine-tuning-attack-for-large","repo_url":"https://github.com/git-disl/virus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"red-teaming","task_name":"Red Teaming"},{"task_slug":"safety-alignment","task_name":"Safety Alignment"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.17433","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.17433"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/git-disl/virus","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":1,"ran":1,"ran_draft_wrong":3,"unverified":6},"by_repo_kind":{"official":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"175778f00233af46","entry":"adapt_dimension_b2a","repo":"git-disl/virus","repo_kind":"official","path":"loss_func/repnoise_loss.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/loss_func/repnoise_loss.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"175778f00233af46"}},{"code_sha256_prefix":"c9b7e712eaf8afe1","entry":"extract_answer_number","repo":"git-disl/virus","repo_kind":"official","path":"gsm8k/pred_eval.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/gsm8k/pred_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c9b7e712eaf8afe1"}},{"code_sha256_prefix":"d07d04439cd1d44f","entry":"jload","repo":"git-disl/virus","repo_kind":"official","path":"datasets_visualizer.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/datasets_visualizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d07d04439cd1d44f"}},{"code_sha256_prefix":"dbc0448520ee9257","entry":"masked_token_ce_loss","repo":"git-disl/virus","repo_kind":"official","path":"loss_func/repnoise_loss.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/loss_func/repnoise_loss.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dbc0448520ee9257"}},{"code_sha256_prefix":"c605efc9460b6c33","entry":"register_activation_hook","repo":"git-disl/virus","repo_kind":"official","path":"loss_func/repnoise_loss.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/loss_func/repnoise_loss.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c605efc9460b6c33"}},{"code_sha256_prefix":"5f4323e1e87dc83b","entry":"calculate_drift2first_embedding","repo":"git-disl/virus","repo_kind":"official","path":"utils.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5f4323e1e87dc83b"}},{"code_sha256_prefix":"f44037b3edbce98a","entry":"find_source_len","repo":"git-disl/virus","repo_kind":"official","path":"trainer.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/trainer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f44037b3edbce98a"}},{"code_sha256_prefix":"88c8a24923c625a1","entry":"get_leaf_modules_with_grad","repo":"git-disl/virus","repo_kind":"official","path":"trainer.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/trainer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"88c8a24923c625a1"}},{"code_sha256_prefix":"fbff253d3f3e0ded","entry":"list_to_dataset","repo":"git-disl/virus","repo_kind":"official","path":"datasets_visualizer.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/datasets_visualizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fbff253d3f3e0ded"}},{"code_sha256_prefix":"d4de99126314df09","entry":"load_suffix","repo":"git-disl/virus","repo_kind":"official","path":"datasets_visualizer.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/datasets_visualizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d4de99126314df09"}},{"code_sha256_prefix":"bda772180d04ada3","entry":"track_embedding","repo":"git-disl/virus","repo_kind":"official","path":"utils.py","file_url":"https://github.com/git-disl/virus/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bda772180d04ada3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}