{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-safety-realignment-framework-via-subspace","title":"A safety realignment framework via subspace-oriented model fusion for large language models","arxiv_id":"2405.09055","date":"2024-05-15","proceeding":null,"authors":["Xin Yi","Shunfan Zheng","LinLin Wang","Xiaoling Wang","Liang He"],"abstract":"The current safeguard mechanisms for large language models (LLMs) are indeed susceptible to jailbreak attacks, making them inherently fragile. Even the process of fine-tuning on apparently benign data for downstream tasks can jeopardize safety. One potential solution is to conduct safety fine-tuning subsequent to downstream fine-tuning. However, there's a risk of catastrophic forgetting during safety fine-tuning, where LLMs may regain safety measures but lose the task-specific knowledge acquired during downstream fine-tuning. In this paper, we introduce a safety realignment framework through subspace-oriented model fusion (SOMF), aiming to combine the safeguard capabilities of initially aligned model and the current fine-tuned model into a realigned model. Our approach begins by disentangling all task vectors from the weights of each fine-tuned model. We then identify safety-related regions within these vectors by subspace masking techniques. Finally, we explore the fusion of the initial safely aligned LLM with all task vectors based on the identified safety subspace. We validate that our safety realignment framework satisfies the safety requirements of a single fine-tuned model as well as multiple models during their fusion. Our findings confirm that SOMF preserves safety without notably compromising performance on downstream tasks, including instruction following in Chinese, English, and Hindi, as well as problem-solving capabilities in Code and Math.","url_abs":"https://arxiv.org/abs/2405.09055v1","url_pdf":"https://arxiv.org/pdf/2405.09055v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-safety-realignment-framework-via-subspace","repo_url":"https://github.com/xinykou/safety_realignment","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"instruction-following","task_name":"Instruction Following"},{"task_slug":"math","task_name":"Math"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.09055","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.09055"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xinykou/safety_realignment","reach":{"status":"ok"}}],"summary":{"ran":7,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"1f0228d9c0918a58","entry":"chat_completion","repo":"xinykou/safety_realignment","repo_kind":"official","path":"llama_factory/evaluation/safety/operations.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/llama_factory/evaluation/safety/operations.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f0228d9c0918a58"}},{"code_sha256_prefix":"2a5e953676a20132","entry":"chat_completion","repo":"xinykou/safety_realignment","repo_kind":"official","path":"evaluate/util.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/evaluate/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a5e953676a20132"}},{"code_sha256_prefix":"6e385117f37f623d","entry":"chat_completion_default","repo":"xinykou/safety_realignment","repo_kind":"official","path":"llama_factory/evaluation/safety/operations.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/llama_factory/evaluation/safety/operations.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e385117f37f623d"}},{"code_sha256_prefix":"e5c822756cbe46fd","entry":"chat_completion_default","repo":"xinykou/safety_realignment","repo_kind":"official","path":"evaluate/util.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/evaluate/util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5c822756cbe46fd"}},{"code_sha256_prefix":"3ab3c57755867f9a","entry":"find_layers","repo":"xinykou/safety_realignment","repo_kind":"official","path":"lm_eval/code_task/modeling.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/lm_eval/code_task/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ab3c57755867f9a"}},{"code_sha256_prefix":"728c60994ca0f8f0","entry":"gen_prompt","repo":"xinykou/safety_realignment","repo_kind":"official","path":"evaluate/safety_datasets.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/evaluate/safety_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"728c60994ca0f8f0"}},{"code_sha256_prefix":"20a7cc804eb22661","entry":"hash_args","repo":"xinykou/safety_realignment","repo_kind":"official","path":"lm_eval/api/model.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/lm_eval/api/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"20a7cc804eb22661"}},{"code_sha256_prefix":"d3c3a36b19860bdc","entry":"random_select_prompt","repo":"xinykou/safety_realignment","repo_kind":"official","path":"llama_factory/evaluation/safety/operations.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/llama_factory/evaluation/safety/operations.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d3c3a36b19860bdc"}},{"code_sha256_prefix":"28e35d0a0fcd8e8b","entry":"load_quant","repo":"xinykou/safety_realignment","repo_kind":"official","path":"lm_eval/code_task/modeling.py","file_url":"https://github.com/xinykou/safety_realignment/blob/HEAD/lm_eval/code_task/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"28e35d0a0fcd8e8b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}