{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/wagle-strategic-weight-attribution-for","title":"WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models","arxiv_id":"2410.17509","date":"2024-10-23","proceeding":null,"authors":["Jinghan Jia","Jiancheng Liu","Yihua Zhang","Parikshit Ram","Nathalie Baracaldo","Sijia Liu"],"abstract":"The need for effective unlearning mechanisms in large language models (LLMs) is increasingly urgent, driven by the necessity to adhere to data regulations and foster ethical generative AI practices. Despite growing interest of LLM unlearning, much of the existing research has focused on varied unlearning method designs to boost effectiveness and efficiency. However, the inherent relationship between model weights and LLM unlearning has not been extensively examined. In this paper, we systematically explore how model weights interact with unlearning processes in LLMs and we design the weight attribution-guided LLM unlearning method, WAGLE, which unveils the interconnections between 'influence' of weights and 'influence' of data to forget and retain in LLM generation. By strategically guiding the LLM unlearning across different types of unlearning methods and tasks, WAGLE can erase the undesired content, while maintaining the performance of the original tasks. We refer to the weight attribution-guided LLM unlearning method as WAGLE, which unveils the interconnections between 'influence' of weights and 'influence' of data to forget and retain in LLM generation. Our extensive experiments show that WAGLE boosts unlearning performance across a range of LLM unlearning methods such as gradient difference and (negative) preference optimization, applications such as fictitious unlearning, malicious use prevention, and copyrighted information removal, and models including Zephyr-7b-beta and Llama2-7b. To the best of our knowledge, our work offers the first principled method for attributing and pinpointing the influential weights in enhancing LLM unlearning. It stands in contrast to previous methods that lack weight attribution and simpler weight attribution techniques.","url_abs":"https://arxiv.org/abs/2410.17509v1","url_pdf":"https://arxiv.org/pdf/2410.17509v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"wagle-strategic-weight-attribution-for","repo_url":"https://github.com/OPTML-Group/WAGLE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"wagle-strategic-weight-attribution-for","repo_url":"https://github.com/optml-group/epic","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"wagle-strategic-weight-attribution-for","repo_url":"https://github.com/optml-group/unlearn-ilu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.17509","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.17509"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/OPTML-Group/WAGLE","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optml-group/unlearn-ilu","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optml-group/epic","reach":{"status":"ok"}}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dee8be97b2cf4692","entry":"get_wikitext2","repo":"OPTML-Group/WAGLE","repo_kind":"official","path":"src/dataset/dataset.py","file_url":"https://github.com/OPTML-Group/WAGLE/blob/HEAD/src/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dee8be97b2cf4692"}},{"code_sha256_prefix":"7630a157a3b32ab9","entry":"get_c4","repo":"OPTML-Group/WAGLE","repo_kind":"official","path":"src/dataset/dataset.py","file_url":"https://github.com/OPTML-Group/WAGLE/blob/HEAD/src/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7630a157a3b32ab9"}},{"code_sha256_prefix":"16994008353af958","entry":"get_loaders","repo":"OPTML-Group/WAGLE","repo_kind":"official","path":"src/dataset/dataset.py","file_url":"https://github.com/OPTML-Group/WAGLE/blob/HEAD/src/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16994008353af958"}},{"code_sha256_prefix":"21b851de3f5356c4","entry":"unlearncollector","repo":"OPTML-Group/WAGLE","repo_kind":"official","path":"src/dataset/Base.py","file_url":"https://github.com/OPTML-Group/WAGLE/blob/HEAD/src/dataset/Base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21b851de3f5356c4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}