{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/boosting-a-model-zoo-for-multi-task-and","title":"Model Zoo: A Growing \"Brain\" That Learns Continually","arxiv_id":"2106.03027","date":"2021-06-06","proceeding":null,"authors":["Rahul Ramesh","Pratik Chaudhari"],"abstract":"This paper argues that continual learning methods can benefit by splitting the capacity of the learner across multiple models. We use statistical learning theory and experimental analysis to show how multiple tasks can interact with each other in a non-trivial fashion when a single model is trained on them. The generalization error on a particular task can improve when it is trained with synergistic tasks, but can also deteriorate when trained with competing tasks. This theory motivates our method named Model Zoo which, inspired from the boosting literature, grows an ensemble of small models, each of which is trained during one episode of continual learning. We demonstrate that Model Zoo obtains large gains in accuracy on a variety of continual learning benchmark problems. Code is available at https://github.com/grasp-lyrl/modelzoo_continual.","url_abs":"https://arxiv.org/abs/2106.03027v3","url_pdf":"https://arxiv.org/pdf/2106.03027v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"boosting-a-model-zoo-for-multi-task-and","repo_url":"https://github.com/grasp-lyrl/modelzoo_continual","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"boosting-a-model-zoo-for-multi-task-and","repo_url":"https://github.com/rahul13ramesh/modelzoo_continual","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"learning-theory","task_name":"Learning Theory"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/continual-learning-on-cifar100-20-tasks","task":"Continual Learning","dataset":"Cifar100 (20 tasks)","model":"Model Zoo-Continual","rank_in_archive_order":1,"of":9,"metrics":{"Average Accuracy":"94.99"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-coarse-cifar100","task":"Continual Learning","dataset":"Coarse-CIFAR100","model":"Model Zoo-Continual","rank_in_archive_order":1,"of":1,"metrics":{"Average Accuracy":"84.27"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-permuted-mnist","task":"Continual Learning","dataset":"Permuted MNIST","model":"Model Zoo-Continual","rank_in_archive_order":2,"of":3,"metrics":{"Average Accuracy":"97.71"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-rotated-mnist","task":"Continual Learning","dataset":"Rotated MNIST","model":"Model Zoo-Continual","rank_in_archive_order":1,"of":1,"metrics":{"Average Accuracy":"99.66"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.03027","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.03027"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rahul13ramesh/modelzoo_continual","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/grasp-lyrl/modelzoo_continual","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c8a33e8446cb2845","entry":"evaluate","repo":"grasp-lyrl/modelzoo_continual","repo_kind":"official","path":"utils/run_net.py","file_url":"https://github.com/grasp-lyrl/modelzoo_continual/blob/HEAD/utils/run_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8a33e8446cb2845"}},{"code_sha256_prefix":"e3878893e6d057d2","entry":"run_epoch","repo":"grasp-lyrl/modelzoo_continual","repo_kind":"official","path":"utils/run_net.py","file_url":"https://github.com/grasp-lyrl/modelzoo_continual/blob/HEAD/utils/run_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3878893e6d057d2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}