{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/posterior-meta-replay-for-continual-learning","title":"Posterior Meta-Replay for Continual Learning","arxiv_id":"2103.01133","date":"2021-03-01","proceeding":"NeurIPS 2021 12","authors":["Christian Henning","Maria R. Cervera","Francesco D'Angelo","Johannes von Oswald","Regina Traber","Benjamin Ehret","Seijin Kobayashi","Benjamin F. Grewe","João Sacramento"],"abstract":"Learning a sequence of tasks without access to i.i.d. observations is a widely studied form of continual learning (CL) that remains challenging. In principle, Bayesian learning directly applies to this setting, since recursive and one-off Bayesian updates yield the same result. In practice, however, recursive updating often leads to poor trade-off solutions across tasks because approximate inference is necessary for most models of interest. Here, we describe an alternative Bayesian approach where task-conditioned parameter distributions are continually inferred from data. We offer a practical deep learning implementation of our framework based on probabilistic task-conditioned hypernetworks, an approach we term posterior meta-replay. Experiments on standard benchmarks show that our probabilistic hypernetworks compress sequences of posterior parameter distributions with virtually no forgetting. We obtain considerable performance gains compared to existing Bayesian CL methods, and identify task inference as our major limiting factor. This limitation has several causes that are independent of the considered sequential setting, opening up new avenues for progress in CL.","url_abs":"https://arxiv.org/abs/2103.01133v3","url_pdf":"https://arxiv.org/pdf/2103.01133v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"posterior-meta-replay-for-continual-learning","repo_url":"https://github.com/chrhenning/posterior_replay_cl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"posterior-meta-replay-for-continual-learning","repo_url":"https://github.com/chrhenning/hypnettorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"posterior-meta-replay-for-continual-learning","repo_url":"https://github.com/pennfranc/hypnettorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.01133","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.01133"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chrhenning/hypnettorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pennfranc/hypnettorch","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chrhenning/posterior_replay_cl","reach":{"status":"unanswered"}}],"summary":{"ran_violates":1,"ran_draft_wrong":3,"unverified":1},"by_repo_kind":{"listed":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"70fa25de9903483e","entry":"_grad_kin_energy","repo":"chrhenning/hypnettorch","repo_kind":"listed","path":"hypnettorch/utils/hmc.py","file_url":"https://github.com/chrhenning/hypnettorch/blob/HEAD/hypnettorch/utils/hmc.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"70fa25de9903483e"}},{"code_sha256_prefix":"055ff336e35c176c","entry":"_grad_pot_energy","repo":"chrhenning/hypnettorch","repo_kind":"listed","path":"hypnettorch/utils/hmc.py","file_url":"https://github.com/chrhenning/hypnettorch/blob/HEAD/hypnettorch/utils/hmc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"055ff336e35c176c"}},{"code_sha256_prefix":"1ac9127221a6c0ee","entry":"_kin_energy","repo":"chrhenning/hypnettorch","repo_kind":"listed","path":"hypnettorch/utils/hmc.py","file_url":"https://github.com/chrhenning/hypnettorch/blob/HEAD/hypnettorch/utils/hmc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1ac9127221a6c0ee"}},{"code_sha256_prefix":"797f157d213d455a","entry":"leapfrog","repo":"chrhenning/hypnettorch","repo_kind":"listed","path":"hypnettorch/utils/hmc.py","file_url":"https://github.com/chrhenning/hypnettorch/blob/HEAD/hypnettorch/utils/hmc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"797f157d213d455a"}},{"code_sha256_prefix":"e9957ee092a605ed","entry":"HMC","repo":"chrhenning/hypnettorch","repo_kind":"listed","path":"hypnettorch/utils/hmc.py","file_url":"https://github.com/chrhenning/hypnettorch/blob/HEAD/hypnettorch/utils/hmc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e9957ee092a605ed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}