{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lifelong-learning-with-dynamically-expandable","title":"Lifelong Learning with Dynamically Expandable Networks","arxiv_id":"1708.01547","date":"2017-08-04","proceeding":"ICLR 2018 1","authors":["Jaehong Yoon","Eunho Yang","Jeongtae Lee","Sung Ju Hwang"],"abstract":"We propose a novel deep network architecture for lifelong learning which we\nrefer to as Dynamically Expandable Network (DEN), that can dynamically decide\nits network capacity as it trains on a sequence of tasks, to learn a compact\noverlapping knowledge sharing structure among tasks. DEN is efficiently trained\nin an online manner by performing selective retraining, dynamically expands\nnetwork capacity upon arrival of each task with only the necessary number of\nunits, and effectively prevents semantic drift by splitting/duplicating units\nand timestamping them. We validate DEN on multiple public datasets under\nlifelong learning scenarios, on which it not only significantly outperforms\nexisting lifelong learning methods for deep networks, but also achieves the\nsame level of performance as the batch counterparts with substantially fewer\nnumber of parameters. Further, the obtained network fine-tuned on all tasks\nobtained significantly better performance over the batch models, which shows\nthat it can be used to estimate the optimal network structure even when all\ntasks are available in the first place.","url_abs":"http://arxiv.org/abs/1708.01547v11","url_pdf":"http://arxiv.org/pdf/1708.01547v11.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lifelong-learning-with-dynamically-expandable","repo_url":"https://github.com/epsilon-deltta/ssd_guillotine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"lifelong-learning-with-dynamically-expandable","repo_url":"https://github.com/mako-anon/mako","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"lifelong-learning-with-dynamically-expandable","repo_url":"https://github.com/jaehong-yoon93/DEN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"lifelong-learning","task_name":"Lifelong learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1708.01547","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}