{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/model-sparsity-can-simplify-machine-1","title":"Model Sparsity Can Simplify Machine Unlearning","arxiv_id":"2304.04934","date":"2023-04-11","proceeding":"NeurIPS 2023 11","authors":["Jinghan Jia","Jiancheng Liu","Parikshit Ram","Yuguang Yao","Gaowen Liu","Yang Liu","Pranay Sharma","Sijia Liu"],"abstract":"In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process to remove the influence of specific examples from a given model. Although exact unlearning can be achieved through complete model retraining using the remaining dataset, the associated computational costs have driven the development of efficient, approximate unlearning techniques. Moving beyond data-centric MU approaches, our study introduces a novel model-based perspective: model sparsification via weight pruning, which is capable of reducing the gap between exact unlearning and approximate unlearning. We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient. This leads to a new MU paradigm, termed prune first, then unlearn, which infuses a sparse model prior into the unlearning process. Building on this insight, we also develop a sparsity-aware unlearning method that utilizes sparsity regularization to enhance the training process of approximate unlearning. Extensive experiments show that our proposals consistently benefit MU in various unlearning scenarios. A notable highlight is the 77% unlearning efficacy gain of fine-tuning (one of the simplest unlearning methods) when using sparsity-aware unlearning. Furthermore, we demonstrate the practical impact of our proposed MU methods in addressing other machine learning challenges, such as defending against backdoor attacks and enhancing transfer learning. Codes are available at https://github.com/OPTML-Group/Unlearn-Sparse.","url_abs":"https://arxiv.org/abs/2304.04934v10","url_pdf":"https://arxiv.org/pdf/2304.04934v10.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-sparsity-can-simplify-machine-1","repo_url":"https://github.com/optml-group/unlearn-sparse","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"machine-unlearning","task_name":"Machine Unlearning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.04934","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.04934"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/OPTML-Group/Unlearn-Sparse","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optml-group/unlearn-sparse","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"ran_honours":2,"unverified":10},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2258f6052be52592","entry":"_get_batch","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT_prune_bi.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT_prune_bi.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2258f6052be52592"}},{"code_sha256_prefix":"9482799ff10081ca","entry":"_iterative_unlearn_impl","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT_prune_bi.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT_prune_bi.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9482799ff10081ca"}},{"code_sha256_prefix":"c25079d6cf22a5e2","entry":"fisher_information_martix","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/optim/unlearn/fisher.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/optim/unlearn/fisher.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c25079d6cf22a5e2"}},{"code_sha256_prefix":"cae29c9fba744465","entry":"l1_regularization","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cae29c9fba744465"}},{"code_sha256_prefix":"eac813c9d8e15efd","entry":"l2_regularization","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eac813c9d8e15efd"}},{"code_sha256_prefix":"e97a7ed8049d7b1d","entry":"FT_iter","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT_prune_bi.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT_prune_bi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e97a7ed8049d7b1d"}},{"code_sha256_prefix":"9f427b83aff6b7ac","entry":"FT_prune_bi","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT_prune_bi.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT_prune_bi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f427b83aff6b7ac"}},{"code_sha256_prefix":"f0c9a29156911331","entry":"accuracy","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/core/utils.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/core/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c9a29156911331"}},{"code_sha256_prefix":"eb78e48639bff7ca","entry":"fisher","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/optim/unlearn/fisher.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/optim/unlearn/fisher.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb78e48639bff7ca"}},{"code_sha256_prefix":"e953ece8832f74dd","entry":"get_mean_var","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/optim/unlearn/fisher.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/optim/unlearn/fisher.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e953ece8832f74dd"}},{"code_sha256_prefix":"e84afab70b6b9461","entry":"iterative_unlearn","repo":"optml-group/unlearn-sparse","repo_kind":"official","path":"src/optim/unlearn/FT_prune_bi.py","file_url":"https://github.com/optml-group/unlearn-sparse/blob/HEAD/src/optim/unlearn/FT_prune_bi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e84afab70b6b9461"}},{"code_sha256_prefix":"9ddeab05b84d103f","entry":"load_checkpoint","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/core/utils.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/core/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ddeab05b84d103f"}},{"code_sha256_prefix":"eb0fc3196c9d681c","entry":"parse_args","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/core/arg_parser.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/core/arg_parser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb0fc3196c9d681c"}},{"code_sha256_prefix":"bf11e731c537c74a","entry":"sam_grad","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/optim/unlearn/Wfisher.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/optim/unlearn/Wfisher.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf11e731c537c74a"}},{"code_sha256_prefix":"83fc8f0570aaf3a5","entry":"woodfisher","repo":"OPTML-Group/Unlearn-Sparse","repo_kind":"official","path":"src/optim/unlearn/Wfisher.py","file_url":"https://github.com/OPTML-Group/Unlearn-Sparse/blob/HEAD/src/optim/unlearn/Wfisher.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83fc8f0570aaf3a5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}