{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/model-surgery-modulating-llm-s-behavior-via","title":"Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing","arxiv_id":"2407.08770","date":"2024-07-11","proceeding":null,"authors":["Huanqian Wang","Yang Yue","Rui Lu","Jingxin Shi","Andrew Zhao","Shenzhi Wang","Shiji Song","Gao Huang"],"abstract":"Large Language Models (LLMs) have demonstrated great potential as generalist assistants, showcasing powerful task understanding and problem-solving capabilities. To deploy LLMs as AI assistants, it is crucial that these models exhibit desirable behavioral traits, such as non-toxicity and resilience against jailbreak attempts. Current methods for detoxification or preventing jailbreaking usually involve Supervised Fine-Tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF), which requires finetuning billions of parameters through gradient descent with substantial computation cost. Furthermore, models modified through SFT and RLHF may deviate from the pretrained models, potentially leading to a degradation in foundational LLM capabilities. In this paper, we observe that surprisingly, directly editing a small subset of parameters can effectively modulate specific behaviors of LLMs, such as detoxification and resistance to jailbreaking. Specifically, for a behavior that we aim to avoid, we employ a linear classifier, which we term the behavior probe, to classify binary behavior labels within the hidden state space of the LLM. Using this probe, we introduce an algorithm to identify a critical subset of LLM parameters that significantly influence this targeted behavior. Then we directly edit these selected parameters by shifting them towards the behavior probe. Such a direct parameter editing method necessitates only inference-level computational resources. Experiments demonstrate that in the representative detoxification task, our approach achieves reductions of up to 90.0\\% in toxicity on the RealToxicityPrompts dataset and 49.2\\% on ToxiGen, while maintaining the LLM's general capabilities in areas such as common sense, question answering, and mathematics. Our code is available at https://github.com/lucywang720/model-surgery.","url_abs":"https://arxiv.org/abs/2407.08770v1","url_pdf":"https://arxiv.org/pdf/2407.08770v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-surgery-modulating-llm-s-behavior-via","repo_url":"https://github.com/lucywang720/model-surgery","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[{"method_slug":"sft","method_name":"SFT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2407.08770","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.08770"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucywang720/model-surgery","reach":{"status":"ok"}}],"summary":{"ran":8,"unverified":5},"by_repo_kind":{"official":{"samples":13,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"9f95f3541586ad5c","entry":"cached_collate_fn","repo":"lucywang720/model-surgery","repo_kind":"official","path":"train/run_pplm_discrim_train.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/train/run_pplm_discrim_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f95f3541586ad5c"}},{"code_sha256_prefix":"cb12ad8df155ef2a","entry":"check_jailbroken","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/jailbreak/score.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/jailbreak/score.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb12ad8df155ef2a"}},{"code_sha256_prefix":"c5e400dbec9871ce","entry":"collate_fn","repo":"lucywang720/model-surgery","repo_kind":"official","path":"train/run_pplm_discrim_train.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/train/run_pplm_discrim_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5e400dbec9871ce"}},{"code_sha256_prefix":"c1515ec6a041a7e6","entry":"evaluate_performance","repo":"lucywang720/model-surgery","repo_kind":"official","path":"train/run_pplm_discrim_train.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/train/run_pplm_discrim_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c1515ec6a041a7e6"}},{"code_sha256_prefix":"0064be5c9154d721","entry":"generate_completions","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0064be5c9154d721"}},{"code_sha256_prefix":"b7b7c929e2921723","entry":"get_json","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/jailbreak/score.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/jailbreak/score.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7b7c929e2921723"}},{"code_sha256_prefix":"295e0b897ef80124","entry":"obtain_stats","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/jailbreak/score.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/jailbreak/score.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"295e0b897ef80124"}},{"code_sha256_prefix":"f1c3821c9446b628","entry":"truncate_after_string","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/gsm/run_eval.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/gsm/run_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1c3821c9446b628"}},{"code_sha256_prefix":"e51c9713fadad16f","entry":"create_prompt_with_llama2_chat_format","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/templates.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/templates.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e51c9713fadad16f"}},{"code_sha256_prefix":"697d92a5d51eff3e","entry":"create_prompt_with_tulu_chat_format","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/templates.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/templates.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"697d92a5d51eff3e"}},{"code_sha256_prefix":"dcad9ac62979ca49","entry":"dispatch_openai_chat_requesets","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/dispatch_openai_requests.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/dispatch_openai_requests.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dcad9ac62979ca49"}},{"code_sha256_prefix":"71e4259026911444","entry":"dispatch_openai_prompt_requesets","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/dispatch_openai_requests.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/dispatch_openai_requests.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"71e4259026911444"}},{"code_sha256_prefix":"d134c88af0236263","entry":"get_next_word_predictions","repo":"lucywang720/model-surgery","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/lucywang720/model-surgery/blob/HEAD/eval/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d134c88af0236263"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}