{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pseudo-rehearsal-for-continual-learning-with","title":"Pseudo-Rehearsal for Continual Learning with Normalizing Flows","arxiv_id":"2007.02443","date":"2020-07-05","proceeding":"ICML Workshop LifelongML 2020 7","authors":["Jary Pomponi","Simone Scardapane","Aurelio Uncini"],"abstract":"Catastrophic forgetting (CF) happens whenever a neural network overwrites past knowledge while being trained on new tasks. Common techniques to handle CF include regularization of the weights (using, e.g., their importance on past tasks), and rehearsal strategies, where the network is constantly re-trained on past data. Generative models have also been applied for the latter, in order to have endless sources of data. In this paper, we propose a novel method that combines the strengths of regularization and generative-based rehearsal approaches. Our generative model consists of a normalizing flow (NF), a probabilistic and invertible neural network, trained on the internal embeddings of the network. By keeping a single NF conditioned on the task, we show that our memory overhead remains constant. In addition, exploiting the invertibility of the NF, we propose a simple approach to regularize the network's embeddings with respect to past tasks. We show that our method performs favorably with respect to state-of-the-art approaches in the literature, with bounded computational power and memory overheads.","url_abs":"https://arxiv.org/abs/2007.02443v4","url_pdf":"https://arxiv.org/pdf/2007.02443v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pseudo-rehearsal-for-continual-learning-with","repo_url":"https://github.com/ispamm/PRER","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.02443","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.02443"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ispamm/PRER","reach":null}],"summary":{"ran":8,"ran_draft_wrong":2,"unverified":11},"by_repo_kind":{"official":{"samples":21,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":21,"samples":[{"code_sha256_prefix":"3fa807854aaa4305","entry":"BatchNorm","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fa807854aaa4305"}},{"code_sha256_prefix":"d43422026fb01a01","entry":"Conditioner","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d43422026fb01a01"}},{"code_sha256_prefix":"83607333c7fc8258","entry":"CouplingLayer","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"83607333c7fc8258"}},{"code_sha256_prefix":"200fa3e04941e661","entry":"Encoder","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"200fa3e04941e661"}},{"code_sha256_prefix":"52cca2a1c5d32c72","entry":"NoneConditioner","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52cca2a1c5d32c72"}},{"code_sha256_prefix":"c28cb8903e6fa746","entry":"Prior","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c28cb8903e6fa746"}},{"code_sha256_prefix":"764a0379bdcee1c3","entry":"Reshape","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"764a0379bdcee1c3"}},{"code_sha256_prefix":"f35198d0074078c5","entry":"Split","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f35198d0074078c5"}},{"code_sha256_prefix":"4eaf1511053dc8a7","entry":"gaussian_nll","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4eaf1511053dc8a7"}},{"code_sha256_prefix":"ad91a710aa97e44c","entry":"reconstruction_loss","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad91a710aa97e44c"}},{"code_sha256_prefix":"9db2c3455ae543fd","entry":"Container","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9db2c3455ae543fd"}},{"code_sha256_prefix":"a23dd769c98cf3a9","entry":"Decoder","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a23dd769c98cf3a9"}},{"code_sha256_prefix":"ce218d15010f7d86","entry":"ExperimentConfig","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce218d15010f7d86"}},{"code_sha256_prefix":"18db7c3472875ddd","entry":"NaiveMethod","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"18db7c3472875ddd"}},{"code_sha256_prefix":"8dcb0d35b0a23a29","entry":"PRERproj","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8dcb0d35b0a23a29"}},{"code_sha256_prefix":"e7d6b7ab3a5ec11f","entry":"Permutation","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e7d6b7ab3a5ec11f"}},{"code_sha256_prefix":"7135c7d62a514fbd","entry":"RNVP","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7135c7d62a514fbd"}},{"code_sha256_prefix":"7959332f2d4a6086","entry":"RNVPBlock","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7959332f2d4a6086"}},{"code_sha256_prefix":"c84d2d298f0a6278","entry":"RNVPLevel","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c84d2d298f0a6278"}},{"code_sha256_prefix":"c6fe6fda8d262a8f","entry":"SequentialFlow","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c6fe6fda8d262a8f"}},{"code_sha256_prefix":"68fa55c2336ed20e","entry":"SingleIncrementalTaskSolver","repo":"ispamm/PRER","repo_kind":"official","path":"continual_ai/cl_strategies/multi_task/prer/PRER.py","file_url":"https://github.com/ispamm/PRER/blob/HEAD/continual_ai/cl_strategies/multi_task/prer/PRER.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"68fa55c2336ed20e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}