{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/can-sensitive-information-be-deleted-from","title":"Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks","arxiv_id":"2309.17410","date":"2023-09-29","proceeding":null,"authors":["Vaidehi Patil","Peter Hase","Mohit Bansal"],"abstract":"Pretrained language models sometimes possess knowledge that we do not wish them to, including memorized personal information and knowledge that could be used to harm people. They can also output toxic or harmful text. To mitigate these safety and informational issues, we propose an attack-and-defense framework for studying the task of deleting sensitive information directly from model weights. We study direct edits to model weights because (1) this approach should guarantee that particular deleted information is never extracted by future prompt attacks, and (2) it should protect against whitebox attacks, which is necessary for making claims about safety/privacy in a setting where publicly available model weights could be used to elicit sensitive information. Our threat model assumes that an attack succeeds if the answer to a sensitive question is located among a set of B generated candidates, based on scenarios where the information would be insecure if the answer is among B candidates. Experimentally, we show that even state-of-the-art model editing methods such as ROME struggle to truly delete factual information from models like GPT-J, as our whitebox and blackbox attacks can recover \"deleted\" information from an edited model 38% of the time. These attacks leverage two key observations: (1) that traces of deleted information can be found in intermediate model hidden states, and (2) that applying an editing method for one question may not delete information across rephrased versions of the question. Finally, we provide new defense methods that protect against some extraction attacks, but we do not find a single universally effective defense method. Our results suggest that truly deleting sensitive information is a tractable but difficult problem, since even relatively low attack success rates have potentially severe societal implications for real-world deployment of language models.","url_abs":"https://arxiv.org/abs/2309.17410v1","url_pdf":"https://arxiv.org/pdf/2309.17410v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"can-sensitive-information-be-deleted-from","repo_url":"https://github.com/vaidehi99/infodeletionattacks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"model-editing","task_name":"Model Editing"}],"methods":[{"method_slug":"rome","method_name":"ROME"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.17410","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.17410"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vaidehi99/infodeletionattacks","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":9,"unverified":5},"by_repo_kind":{"official":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"92f6c49ecd85ec5b","entry":"blocks_input_locator","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/hooks.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/hooks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92f6c49ecd85ec5b"}},{"code_sha256_prefix":"c2fd2fc3dcf6a037","entry":"collect_logits","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/plotting.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/plotting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2fd2fc3dcf6a037"}},{"code_sha256_prefix":"545f704e0ca27e96","entry":"final_layernorm_locator","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/hooks.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/hooks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"545f704e0ca27e96"}},{"code_sha256_prefix":"7aa7d15b17e07438","entry":"get_child_module_by_names","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/util/module_utils.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/util/module_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7aa7d15b17e07438"}},{"code_sha256_prefix":"ce37c1e5ca45da4c","entry":"get_value_at_preds","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/plotting.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/plotting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce37c1e5ca45da4c"}},{"code_sha256_prefix":"5320ade14843dae2","entry":"huggingface_model_local_paths","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/util/tfm_utils.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/util/tfm_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5320ade14843dae2"}},{"code_sha256_prefix":"ab1cdc57445d3f76","entry":"make_decoder","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/hooks.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/hooks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab1cdc57445d3f76"}},{"code_sha256_prefix":"78e83f431d38e8bb","entry":"normalize_inconsistent_state_dict_keys","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/util/tfm_utils.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/util/tfm_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78e83f431d38e8bb"}},{"code_sha256_prefix":"b639e905c59033d4","entry":"postprocess_logits","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/plotting.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/plotting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b639e905c59033d4"}},{"code_sha256_prefix":"390ee5748e50d330","entry":"get_leaf_modules","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/util/module_utils.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/util/module_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"390ee5748e50d330"}},{"code_sha256_prefix":"e759667d86453174","entry":"get_local_path_from_huggingface_cdn","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/util/tfm_utils.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/util/tfm_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e759667d86453174"}},{"code_sha256_prefix":"a444132567003b3e","entry":"low_memory_from_pretrained","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/low_memory/enable.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/low_memory/enable.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a444132567003b3e"}},{"code_sha256_prefix":"ad9a1b60539e7185","entry":"low_memory_load","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/low_memory/load.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/low_memory/load.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad9a1b60539e7185"}},{"code_sha256_prefix":"03f5b2a1c286c0a7","entry":"make_layer_names","repo":"vaidehi99/infodeletionattacks","repo_kind":"official","path":"transformer_utils/src/transformer_utils/logit_lens/layer_names.py","file_url":"https://github.com/vaidehi99/infodeletionattacks/blob/HEAD/transformer_utils/src/transformer_utils/logit_lens/layer_names.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03f5b2a1c286c0a7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}