{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safety-layers-of-aligned-large-language","title":"Safety Layers in Aligned Large Language Models: The Key to LLM Security","arxiv_id":"2408.17003","date":"2024-08-30","proceeding":null,"authors":["Shen Li","Liuyi Yao","Lan Zhang","Yaliang Li"],"abstract":"Aligned LLMs are secure, capable of recognizing and refusing to answer malicious questions. However, the role of internal parameters in maintaining such security is not well understood yet, further these models can be vulnerable to security degradation when subjected to fine-tuning attacks. To address these challenges, our work uncovers the mechanism behind security in aligned LLMs at the parameter level, identifying a small set of contiguous layers in the middle of the model that are crucial for distinguishing malicious queries from normal ones, referred to as ``safety layers\". We first confirm the existence of these safety layers by analyzing variations in input vectors within the model's internal layers. Additionally, we leverage the over-rejection phenomenon and parameters scaling analysis to precisely locate the safety layers. Building on these findings, we propose a novel fine-tuning approach, Safely Partial-Parameter Fine-Tuning (SPPFT), that fixes the gradient of the safety layers during fine-tuning to address the security degradation. Our experiments demonstrate that the proposed approach can significantly preserve LLM security while maintaining performance and reducing computational resources compared to full fine-tuning.","url_abs":"https://arxiv.org/abs/2408.17003v5","url_pdf":"https://arxiv.org/pdf/2408.17003v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safety-layers-of-aligned-large-language","repo_url":"https://github.com/listen0425/safety-layers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2408.17003","atlas_url":"https://app.syntology.ai/?focus=2408.17003","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.17003"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/listen0425/safety-layers","reach":null}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"a52ae3f0a1945601","entry":"Prompter","repo":"listen0425/safety-layers","repo_kind":"official","path":"Code/Fine_tune/SPPFT.py","file_url":"https://github.com/listen0425/safety-layers/blob/HEAD/Code/Fine_tune/SPPFT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a52ae3f0a1945601"}},{"code_sha256_prefix":"48a724d59e3a82f5","entry":"smart_tokenizer_and_embedding_resize","repo":"listen0425/safety-layers","repo_kind":"official","path":"Code/Fine_tune/SPPFT.py","file_url":"https://github.com/listen0425/safety-layers/blob/HEAD/Code/Fine_tune/SPPFT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"48a724d59e3a82f5"}},{"code_sha256_prefix":"afead4d15856351f","entry":"train","repo":"listen0425/safety-layers","repo_kind":"official","path":"Code/Fine_tune/SPPFT.py","file_url":"https://github.com/listen0425/safety-layers/blob/HEAD/Code/Fine_tune/SPPFT.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"afead4d15856351f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}