{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/progress-compress-a-scalable-framework-for","title":"Progress & Compress: A scalable framework for continual learning","arxiv_id":"1805.06370","date":"2018-05-16","proceeding":"ICML 2018 7","authors":["Jonathan Schwarz","Jelena Luketina","Wojciech M. Czarnecki","Agnieszka Grabska-Barwinska","Yee Whye Teh","Razvan Pascanu","Raia Hadsell"],"abstract":"We introduce a conceptually simple and scalable framework for continual\nlearning domains where tasks are learned sequentially. Our method is constant\nin the number of parameters and is designed to preserve performance on\npreviously encountered tasks while accelerating learning progress on subsequent\nproblems. This is achieved by training a network with two components: A\nknowledge base, capable of solving previously encountered problems, which is\nconnected to an active column that is employed to efficiently learn the current\ntask. After learning a new task, the active column is distilled into the\nknowledge base, taking care to protect any previously acquired skills. This\ncycle of active learning (progression) followed by consolidation (compression)\nrequires no architecture growth, no access to or storing of previous data or\ntasks, and no task-specific parameters. We demonstrate the progress & compress\napproach on sequential classification of handwritten alphabets as well as two\nreinforcement learning domains: Atari games and 3D maze navigation.","url_abs":"http://arxiv.org/abs/1805.06370v2","url_pdf":"http://arxiv.org/pdf/1805.06370v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"progress-compress-a-scalable-framework-for","repo_url":"https://github.com/wabbajack1/tapd","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"},{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[{"method_slug":"ewc","method_name":"EWC"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.06370","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}