{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learn-unlearn-and-relearn-an-online-learning","title":"Learn, Unlearn and Relearn: An Online Learning Paradigm for Deep Neural Networks","arxiv_id":"2303.10455","date":"2023-03-18","proceeding":null,"authors":["Vijaya Raghavan T. Ramkumar","Elahe Arani","Bahram Zonooz"],"abstract":"Deep neural networks (DNNs) are often trained on the premise that the complete training data set is provided ahead of time. However, in real-world scenarios, data often arrive in chunks over time. This leads to important considerations about the optimal strategy for training DNNs, such as whether to fine-tune them with each chunk of incoming data (warm-start) or to retrain them from scratch with the entire corpus of data whenever a new chunk is available. While employing the latter for training can be resource-intensive, recent work has pointed out the lack of generalization in warm-start models. Therefore, to strike a balance between efficiency and generalization, we introduce Learn, Unlearn, and Relearn (LURE) an online learning paradigm for DNNs. LURE interchanges between the unlearning phase, which selectively forgets the undesirable information in the model through weight reinitialization in a data-dependent manner, and the relearning phase, which emphasizes learning on generalizable features. We show that our training paradigm provides consistent performance gains across datasets in both classification and few-shot settings. We further show that it leads to more robust and well-calibrated models.","url_abs":"https://arxiv.org/abs/2303.10455v1","url_pdf":"https://arxiv.org/pdf/2303.10455v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learn-unlearn-and-relearn-an-online-learning","repo_url":"https://github.com/neurai-lab/lure","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2303.10455","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.10455"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neurai-lab/lure","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"42536426e6958f76","entry":"fc_reinitialize","repo":"neurai-lab/lure","repo_kind":"official","path":"LURE_main.py","file_url":"https://github.com/neurai-lab/lure/blob/HEAD/LURE_main.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42536426e6958f76"}},{"code_sha256_prefix":"a09ea2c640f9a668","entry":"reparameterize_non_sparse","repo":"neurai-lab/lure","repo_kind":"official","path":"LURE_main.py","file_url":"https://github.com/neurai-lab/lure/blob/HEAD/LURE_main.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a09ea2c640f9a668"}},{"code_sha256_prefix":"f0c9a29156911331","entry":"accuracy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f0c9a29156911331"}},{"code_sha256_prefix":"95eae63e681452ed","entry":"ll_reinitialize","repo":"neurai-lab/lure","repo_kind":"official","path":"LURE_main.py","file_url":"https://github.com/neurai-lab/lure/blob/HEAD/LURE_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95eae63e681452ed"}},{"code_sha256_prefix":"c101adc56d97d818","entry":"train","repo":"neurai-lab/lure","repo_kind":"official","path":"ALMA.py","file_url":"https://github.com/neurai-lab/lure/blob/HEAD/ALMA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c101adc56d97d818"}},{"code_sha256_prefix":"18fa0b46dca461ea","entry":"validate","repo":"neurai-lab/lure","repo_kind":"official","path":"ALMA.py","file_url":"https://github.com/neurai-lab/lure/blob/HEAD/ALMA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18fa0b46dca461ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}