{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/temporal-difference-variational-continual","title":"Temporal-Difference Variational Continual Learning","arxiv_id":"2410.07812","date":"2024-10-10","proceeding":null,"authors":["Luckeciano C. Melo","Alessandro Abate","Yarin Gal"],"abstract":"Machine Learning models in real-world applications must continuously learn new tasks to adapt to shifts in the data-generating distribution. Yet, for Continual Learning (CL), models often struggle to balance learning new tasks (plasticity) with retaining previous knowledge (memory stability). Consequently, they are susceptible to Catastrophic Forgetting, which degrades performance and undermines the reliability of deployed systems. In the Bayesian CL literature, variational methods tackle this challenge by employing a learning objective that recursively updates the posterior distribution while constraining it to stay close to its previous estimate. Nonetheless, we argue that these methods may be ineffective due to compounding approximation errors over successive recursions. To mitigate this, we propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations, preventing individual errors from dominating future posterior updates and compounding over time. We reveal insightful connections between these objectives and Temporal-Difference methods, a popular learning mechanism in Reinforcement Learning and Neuroscience. Experiments on challenging CL benchmarks show that our approach effectively mitigates Catastrophic Forgetting, outperforming strong Variational CL methods.","url_abs":"https://arxiv.org/abs/2410.07812v2","url_pdf":"https://arxiv.org/pdf/2410.07812v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"temporal-difference-variational-continual","repo_url":"https://github.com/luckeciano/TD-VCL","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.07812","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.07812"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/luckeciano/TD-VCL","reach":null}],"summary":{"ran":3,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"ea8c9588c27bc588","entry":"MeanFieldGaussian","repo":"luckeciano/TD-VCL","repo_kind":"official","path":"modules/td_vcl.py","file_url":"https://github.com/luckeciano/TD-VCL/blob/HEAD/modules/td_vcl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ea8c9588c27bc588"}},{"code_sha256_prefix":"8da53a87921ea3da","entry":"NStepKLVCLBayesianLinear","repo":"luckeciano/TD-VCL","repo_kind":"official","path":"modules/td_vcl.py","file_url":"https://github.com/luckeciano/TD-VCL/blob/HEAD/modules/td_vcl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8da53a87921ea3da"}},{"code_sha256_prefix":"fbcb81fd2645b872","entry":"TDBayesianLinear","repo":"luckeciano/TD-VCL","repo_kind":"official","path":"modules/td_vcl.py","file_url":"https://github.com/luckeciano/TD-VCL/blob/HEAD/modules/td_vcl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbcb81fd2645b872"}},{"code_sha256_prefix":"cbb32079ed8b32f4","entry":"get_activation_function","repo":"luckeciano/TD-VCL","repo_kind":"official","path":"modules/td_vcl.py","file_url":"https://github.com/luckeciano/TD-VCL/blob/HEAD/modules/td_vcl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cbb32079ed8b32f4"}},{"code_sha256_prefix":"62c9960fe899b0ff","entry":"NStepKLVCL","repo":"luckeciano/TD-VCL","repo_kind":"official","path":"modules/td_vcl.py","file_url":"https://github.com/luckeciano/TD-VCL/blob/HEAD/modules/td_vcl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62c9960fe899b0ff"}},{"code_sha256_prefix":"24075c8cb6ae7fb0","entry":"TemporalDifferenceVCL","repo":"luckeciano/TD-VCL","repo_kind":"official","path":"modules/td_vcl.py","file_url":"https://github.com/luckeciano/TD-VCL/blob/HEAD/modules/td_vcl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"24075c8cb6ae7fb0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}