{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zero-shot-machine-unlearning-at-scale-via","title":"An Information Theoretic Approach to Machine Unlearning","arxiv_id":"2402.01401","date":"2024-02-02","proceeding":null,"authors":["Jack Foster","Kyle Fogarty","Stefan Schoepf","Zack Dugue","Cengiz Öztireli","Alexandra Brintrup"],"abstract":"To comply with AI and data regulations, the need to forget private or copyrighted information from trained machine learning models is increasingly important. The key challenge in unlearning is forgetting the necessary data in a timely manner, while preserving model performance. In this work, we address the zero-shot unlearning scenario, whereby an unlearning algorithm must be able to remove data given only a trained model and the data to be forgotten. We explore unlearning from an information theoretic perspective, connecting the influence of a sample to the information gain a model receives by observing it. From this, we derive a simple but principled zero-shot unlearning method based on the geometry of the model. Our approach takes the form of minimising the gradient of a learned function with respect to a small neighbourhood around a target forget point. This induces a smoothing effect, causing forgetting by moving the boundary of the classifier. We explore the intuition behind why this approach can jointly unlearn forget samples while preserving general model performance through a series of low-dimensional experiments. We perform extensive empirical evaluation of our method over a range of contemporary benchmarks, verifying that our method is competitive with state-of-the-art performance under the strict constraints of zero-shot unlearning. Code for the project can be found at https://github.com/jwf40/Information-Theoretic-Unlearning","url_abs":"https://arxiv.org/abs/2402.01401v4","url_pdf":"https://arxiv.org/pdf/2402.01401v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zero-shot-machine-unlearning-at-scale-via","repo_url":"https://github.com/jwf40/information-theoretic-unlearning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"zero-shot-machine-unlearning-at-scale-via","repo_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"machine-unlearning","task_name":"Machine Unlearning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.01401","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.01401"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jwf40/information-theoretic-unlearning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jwf40/zeroshot-unlearning-at-scale","reach":{"status":"ok"}}],"summary":{"ran":10,"ran_draft_wrong":1,"ran_honours":3,"unverified":1},"by_repo_kind":{"official":{"samples":15,"ran":14,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"65e6e724245e6071","entry":"JSDiv","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"65e6e724245e6071"}},{"code_sha256_prefix":"e982685418b19915","entry":"UnLearningScore","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e982685418b19915"}},{"code_sha256_prefix":"35a220c6eeea45d4","entry":"UnlearnerLoss","repo":"jwf40/information-theoretic-unlearning","repo_kind":"official","path":"src/unlearn.py","file_url":"https://github.com/jwf40/information-theoretic-unlearning/blob/HEAD/src/unlearn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35a220c6eeea45d4"}},{"code_sha256_prefix":"b817aa66879a905d","entry":"attention","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/gkt.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/gkt.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b817aa66879a905d"}},{"code_sha256_prefix":"fa2c8ba3cdf9f1d6","entry":"attention_diff","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/gkt.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/gkt.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa2c8ba3cdf9f1d6"}},{"code_sha256_prefix":"1142b442d4f39943","entry":"build_retain_forget_sets","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/forget_full_class_strategies.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/forget_full_class_strategies.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1142b442d4f39943"}},{"code_sha256_prefix":"ccc32fcda24fa317","entry":"build_retain_forget_sets","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/forget_subclass_strategies.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/forget_subclass_strategies.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ccc32fcda24fa317"}},{"code_sha256_prefix":"391f84dc0ccdbf82","entry":"divergence","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/gkt.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/gkt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"391f84dc0ccdbf82"}},{"code_sha256_prefix":"4a12bd3434683bbc","entry":"entropy","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a12bd3434683bbc"}},{"code_sha256_prefix":"ea3bbc7bfbf7ad33","entry":"get_classwise_ds","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/forget_full_class_strategies.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/forget_full_class_strategies.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ea3bbc7bfbf7ad33"}},{"code_sha256_prefix":"93763b90cbaa67db","entry":"get_classwise_ds","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/forget_subclass_strategies.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/forget_subclass_strategies.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"93763b90cbaa67db"}},{"code_sha256_prefix":"01df4dbd4f3784f4","entry":"get_gf_loader","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/forget_random_strategies.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/forget_random_strategies.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01df4dbd4f3784f4"}},{"code_sha256_prefix":"ddad3b26002d7f99","entry":"guided_eval","repo":"jwf40/zeroshot-unlearning-at-scale","repo_kind":"official","path":"src/forget_random_strategies.py","file_url":"https://github.com/jwf40/zeroshot-unlearning-at-scale/blob/HEAD/src/forget_random_strategies.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ddad3b26002d7f99"}},{"code_sha256_prefix":"7920183b7b2256da","entry":"unlearning_step","repo":"jwf40/information-theoretic-unlearning","repo_kind":"official","path":"src/unlearn.py","file_url":"https://github.com/jwf40/information-theoretic-unlearning/blob/HEAD/src/unlearn.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7920183b7b2256da"}},{"code_sha256_prefix":"015561a0364e1ddb","entry":"fit_one_unlearning_cycle","repo":"jwf40/information-theoretic-unlearning","repo_kind":"official","path":"src/unlearn.py","file_url":"https://github.com/jwf40/information-theoretic-unlearning/blob/HEAD/src/unlearn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"015561a0364e1ddb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}