{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/in-context-unlearning-language-models-as-few","title":"In-Context Unlearning: Language Models as Few Shot Unlearners","arxiv_id":"2310.07579","date":"2023-10-11","proceeding":null,"authors":["Martin Pawelczyk","Seth Neel","Himabindu Lakkaraju"],"abstract":"Machine unlearning, the study of efficiently removing the impact of specific training instances on a model, has garnered increased attention in recent years due to regulatory guidelines such as the \\emph{Right to be Forgotten}. Achieving precise unlearning typically involves fully retraining the model and is computationally infeasible in case of very large models such as Large Language Models (LLMs). To this end, recent work has proposed several algorithms which approximate the removal of training data without retraining the model. These algorithms crucially rely on access to the model parameters in order to update them, an assumption that may not hold in practice due to computational constraints or having only query access to the LLMs. In this work, we propose a new class of unlearning methods for LLMs called ``In-Context Unlearning.'' This method unlearns instances from the model by simply providing specific kinds of inputs in context, without the need to update model parameters. To unlearn specific training instances, we present these instances to the LLMs at inference time along with labels that differ from their ground truth. Our experimental results demonstrate that in-context unlearning performs on par with, or in some cases outperforms other state-of-the-art methods that require access to model parameters, effectively removing the influence of specific instances on the model while preserving test accuracy.","url_abs":"https://arxiv.org/abs/2310.07579v4","url_pdf":"https://arxiv.org/pdf/2310.07579v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"in-context-unlearning-language-models-as-few","repo_url":"https://github.com/MartinPawel/In-Context-Unlearning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"in-context-unlearning-language-models-as-few","repo_url":"https://github.com/martinpawel/openunlearn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"machine-unlearning","task_name":"Machine Unlearning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.07579","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.07579"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MartinPawel/In-Context-Unlearning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/martinpawel/openunlearn","reach":null}],"summary":{"ran":2,"ran_fixture":1,"ran_draft_wrong":1,"ran_honours":2,"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"7b64c6ac31cd548d","entry":"NGP","repo":"martinpawel/openunlearn","repo_kind":"listed","path":"openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","file_url":"https://github.com/martinpawel/openunlearn/blob/HEAD/openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b64c6ac31cd548d"}},{"code_sha256_prefix":"42702605d91b2e0d","entry":"compute_causal_loss","repo":"martinpawel/openunlearn","repo_kind":"listed","path":"openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","file_url":"https://github.com/martinpawel/openunlearn/blob/HEAD/openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42702605d91b2e0d"}},{"code_sha256_prefix":"12d96eb95bdf533c","entry":"compute_classification_loss","repo":"martinpawel/openunlearn","repo_kind":"listed","path":"openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","file_url":"https://github.com/martinpawel/openunlearn/blob/HEAD/openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"12d96eb95bdf533c"}},{"code_sha256_prefix":"4c5b12c40410e08f","entry":"eval_llm_model","repo":"martinpawel/openunlearn","repo_kind":"listed","path":"openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","file_url":"https://github.com/martinpawel/openunlearn/blob/HEAD/openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c5b12c40410e08f"}},{"code_sha256_prefix":"67b0cd2c25da8b69","entry":"eval_model","repo":"martinpawel/openunlearn","repo_kind":"listed","path":"openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","file_url":"https://github.com/martinpawel/openunlearn/blob/HEAD/openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67b0cd2c25da8b69"}},{"code_sha256_prefix":"c18ab10945f72e66","entry":"select_short_reviews","repo":"MartinPawel/In-Context-Unlearning","repo_kind":"official","path":"local_dataset_utilities.py","file_url":"https://github.com/MartinPawel/In-Context-Unlearning/blob/HEAD/local_dataset_utilities.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c18ab10945f72e66"}},{"code_sha256_prefix":"7c48fd1e3dde37c5","entry":"Unlearner","repo":"martinpawel/openunlearn","repo_kind":"listed","path":"openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","file_url":"https://github.com/martinpawel/openunlearn/blob/HEAD/openunlearn/unlearners/catalog/neg_grad_plus/neg_grad_plus.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7c48fd1e3dde37c5"}},{"code_sha256_prefix":"b8a02b31356a1521","entry":"clean_dataset","repo":"MartinPawel/In-Context-Unlearning","repo_kind":"official","path":"local_dataset_utilities.py","file_url":"https://github.com/MartinPawel/In-Context-Unlearning/blob/HEAD/local_dataset_utilities.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8a02b31356a1521"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}