{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/continual-learning-with-deep-generative","title":"Continual Learning with Deep Generative Replay","arxiv_id":"1705.08690","date":"2017-05-24","proceeding":"NeurIPS 2017 12","authors":["Hanul Shin","Jung Kwon Lee","Jaehong Kim","Jiwon Kim"],"abstract":"Attempts to train a comprehensive artificial intelligence capable of solving\nmultiple tasks have been impeded by a chronic problem called catastrophic\nforgetting. Although simply replaying all previous data alleviates the problem,\nit requires large memory and even worse, often infeasible in real world\napplications where the access to past data is limited. Inspired by the\ngenerative nature of hippocampus as a short-term memory system in primate\nbrain, we propose the Deep Generative Replay, a novel framework with a\ncooperative dual model architecture consisting of a deep generative model\n(\"generator\") and a task solving model (\"solver\"). With only these two models,\ntraining data for previous tasks can easily be sampled and interleaved with\nthose for a new task. We test our methods in several sequential learning\nsettings involving image classification tasks.","url_abs":"http://arxiv.org/abs/1705.08690v3","url_pdf":"http://arxiv.org/pdf/1705.08690v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"continual-learning-with-deep-generative","repo_url":"https://github.com/epsilon-deltta/ssd_guillotine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"continual-learning-with-deep-generative","repo_url":"https://github.com/geox-lab/cmn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"continual-learning-with-deep-generative","repo_url":"https://github.com/hursung1/DeepGenerativeReplay","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"continual-learning-with-deep-generative","repo_url":"https://github.com/kuc2477/pytorch-deep-generative-replay","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"continual-learning-with-deep-generative","repo_url":"https://github.com/ContinualAI/avalanche","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":null,"task_name":"Hippocampus"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"wgan-gp","method_name":"WGAN GP"},{"method_slug":"wgan-gp-loss","method_name":"WGAN-GP Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.08690","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}