{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/alleviating-catastrophic-forgetting-using","title":"Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization","arxiv_id":"1802.01569","date":"2018-02-02","proceeding":null,"authors":["Nicolas Y. Masse","Gregory D. Grant","David J. Freedman"],"abstract":"Humans and most animals can learn new tasks without forgetting old ones.\nHowever, training artificial neural networks (ANNs) on new tasks typically\ncause it to forget previously learned tasks. This phenomenon is the result of\n\"catastrophic forgetting\", in which training an ANN disrupts connection weights\nthat were important for solving previous tasks, degrading task performance.\nSeveral recent studies have proposed methods to stabilize connection weights of\nANNs that are deemed most important for solving a task, which helps alleviate\ncatastrophic forgetting. Here, drawing inspiration from algorithms that are\nbelieved to be implemented in vivo, we propose a complementary method: adding a\ncontext-dependent gating signal, such that only sparse, mostly non-overlapping\npatterns of units are active for any one task. This method is easy to\nimplement, requires little computational overhead, and allows ANNs to maintain\nhigh performance across large numbers of sequentially presented tasks when\ncombined with weight stabilization. This work provides another example of how\nneuroscience-inspired algorithms can benefit ANN design and capability.","url_abs":"http://arxiv.org/abs/1802.01569v2","url_pdf":"http://arxiv.org/pdf/1802.01569v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"alleviating-catastrophic-forgetting-using","repo_url":"https://github.com/nmasse/Context-Dependent-Gating","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"alleviating-catastrophic-forgetting-using","repo_url":"https://github.com/freedmanlab/Context-Dependent-Gating","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"alleviating-catastrophic-forgetting-using","repo_url":"https://github.com/rhezab/Context-Dependent-Gating-Copy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1802.01569","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}