{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-continually-learn-with-the","title":"Learning to Continually Learn with the Bayesian Principle","arxiv_id":"2405.18758","date":"2024-05-29","proceeding":null,"authors":["Soochan Lee","Hyeonseong Jeon","Jaehyeon Son","Gunhee Kim"],"abstract":"In the present era of deep learning, continual learning research is mainly focused on mitigating forgetting when training a neural network with stochastic gradient descent on a non-stationary stream of data. On the other hand, in the more classical literature of statistical machine learning, many models have sequential Bayesian update rules that yield the same learning outcome as the batch training, i.e., they are completely immune to catastrophic forgetting. However, they are often overly simple to model complex real-world data. In this work, we adopt the meta-learning paradigm to combine the strong representational power of neural networks and simple statistical models' robustness to forgetting. In our novel meta-continual learning framework, continual learning takes place only in statistical models via ideal sequential Bayesian update rules, while neural networks are meta-learned to bridge the raw data and the statistical models. Since the neural networks remain fixed during continual learning, they are protected from catastrophic forgetting. This approach not only achieves significantly improved performance but also exhibits excellent scalability. Since our approach is domain-agnostic and model-agnostic, it can be applied to a wide range of problems and easily integrated with existing model architectures.","url_abs":"https://arxiv.org/abs/2405.18758v1","url_pdf":"https://arxiv.org/pdf/2405.18758v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-continually-learn-with-the","repo_url":"https://github.com/soochan-lee/sb-mcl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.18758","atlas_url":"https://app.syntology.ai/?focus=2405.18758","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18758"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/soochan-lee/SB-MCL","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/soochan-lee/sb-mcl","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"ran":5,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":7,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"2c6bc30a15479d82","entry":"get_continual_mask","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"models/continual_transformer.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/models/continual_transformer.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c6bc30a15479d82"}},{"code_sha256_prefix":"87a56660bd4c8a47","entry":"get_y","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87a56660bd4c8a47"}},{"code_sha256_prefix":"43e280293e440b88","entry":"process_gnt","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"43e280293e440b88"}},{"code_sha256_prefix":"20f1b95de7047461","entry":"resize_image","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"dataset.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"20f1b95de7047461"}},{"code_sha256_prefix":"77813c4947b36c1d","entry":"sample_test_attachment","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"models/continual_transformer.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/models/continual_transformer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77813c4947b36c1d"}},{"code_sha256_prefix":"9fe72d2912ab2734","entry":"sequential_bayes","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"models/sbmcl.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/models/sbmcl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9fe72d2912ab2734"}},{"code_sha256_prefix":"2c8587cbab576d85","entry":"slice_shots","repo":"soochan-lee/SB-MCL","repo_kind":"official","path":"models/sbmcl.py","file_url":"https://github.com/soochan-lee/SB-MCL/blob/HEAD/models/sbmcl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c8587cbab576d85"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}