{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unrolling-sgd-understanding-factors","title":"Unrolling SGD: Understanding Factors Influencing Machine Unlearning","arxiv_id":"2109.13398","date":"2021-09-27","proceeding":null,"authors":["Anvith Thudi","Gabriel Deza","Varun Chandrasekaran","Nicolas Papernot"],"abstract":"Machine unlearning is the process through which a deployed machine learning model is made to forget about some of its training data points. While naively retraining the model from scratch is an option, it is almost always associated with large computational overheads for deep learning models. Thus, several approaches to approximately unlearn have been proposed along with corresponding metrics that formalize what it means for a model to forget about a data point. In this work, we first taxonomize approaches and metrics of approximate unlearning. As a result, we identify verification error, i.e., the L2 difference between the weights of an approximately unlearned and a naively retrained model, as an approximate unlearning metric that should be optimized for as it subsumes a large class of other metrics. We theoretically analyze the canonical training algorithm, stochastic gradient descent (SGD), to surface the variables which are relevant to reducing the verification error of approximate unlearning for SGD. From this analysis, we first derive an easy-to-compute proxy for verification error (termed unlearning error). The analysis also informs the design of a new training objective penalty that limits the overall change in weights during SGD and as a result facilitates approximate unlearning with lower verification error. We validate our theoretical work through an empirical evaluation on learning with CIFAR-10, CIFAR-100, and IMDB sentiment analysis.","url_abs":"https://arxiv.org/abs/2109.13398v2","url_pdf":"https://arxiv.org/pdf/2109.13398v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unrolling-sgd-understanding-factors","repo_url":"https://github.com/cleverhans-lab/unrolling-sgd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"machine-unlearning","task_name":"Machine Unlearning"},{"task_slug":"unrolling","task_name":"Rolling Shutter Correction"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.13398","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.13398"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cleverhans-lab/unrolling-sgd","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":10},"by_repo_kind":{"official":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"860c6eec0095faf2","entry":"Mentr","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"Privacy_risk_score/calculate_PRS.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/Privacy_risk_score/calculate_PRS.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"860c6eec0095faf2"}},{"code_sha256_prefix":"3e93275f5bb6e23e","entry":"ResNet18","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"ResNet_CIFAR10.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/ResNet_CIFAR10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e93275f5bb6e23e"}},{"code_sha256_prefix":"9a28296ebc73a658","entry":"ResNet34","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"ResNet_CIFAR10.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/ResNet_CIFAR10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a28296ebc73a658"}},{"code_sha256_prefix":"479285c5bc2b40ba","entry":"ResNet50","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"ResNet_CIFAR10.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/ResNet_CIFAR10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"479285c5bc2b40ba"}},{"code_sha256_prefix":"63ae7db4794f4b09","entry":"create_data_loader","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"BERT/bert_helpers.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/BERT/bert_helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63ae7db4794f4b09"}},{"code_sha256_prefix":"1341a64904d65de6","entry":"prob_bins","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"Privacy_risk_score/calculate_PRS.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/Privacy_risk_score/calculate_PRS.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1341a64904d65de6"}},{"code_sha256_prefix":"76a2d8f814f3f8ba","entry":"safe_log","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"Privacy_risk_score/calculate_PRS.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/Privacy_risk_score/calculate_PRS.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76a2d8f814f3f8ba"}},{"code_sha256_prefix":"4068f9af45024c22","entry":"validate","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"Privacy_risk_score/prs_vs_ver.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/Privacy_risk_score/prs_vs_ver.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4068f9af45024c22"}},{"code_sha256_prefix":"64873e8a11684e38","entry":"validate","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"Privacy_risk_score/train_shadow_model.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/Privacy_risk_score/train_shadow_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64873e8a11684e38"}},{"code_sha256_prefix":"30ac71b95fafa32e","entry":"weights_to_list_fast","repo":"cleverhans-lab/unrolling-sgd","repo_kind":"official","path":"Privacy_risk_score/Correlation_PRS.py","file_url":"https://github.com/cleverhans-lab/unrolling-sgd/blob/HEAD/Privacy_risk_score/Correlation_PRS.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30ac71b95fafa32e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}