{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/coscl-cooperation-of-small-continual-learners","title":"CoSCL: Cooperation of Small Continual Learners is Stronger than a Big One","arxiv_id":"2207.06543","date":"2022-07-13","proceeding":null,"authors":["Liyuan Wang","Xingxing Zhang","Qian Li","Jun Zhu","Yi Zhong"],"abstract":"Continual learning requires incremental compatibility with a sequence of tasks. However, the design of model architecture remains an open question: In general, learning all tasks with a shared set of parameters suffers from severe interference between tasks; while learning each task with a dedicated parameter subspace is limited by scalability. In this work, we theoretically analyze the generalization errors for learning plasticity and memory stability in continual learning, which can be uniformly upper-bounded by (1) discrepancy between task distributions, (2) flatness of loss landscape and (3) cover of parameter space. Then, inspired by the robust biological learning system that processes sequential experiences with multiple parallel compartments, we propose Cooperation of Small Continual Learners (CoSCL) as a general strategy for continual learning. Specifically, we present an architecture with a fixed number of narrower sub-networks to learn all incremental tasks in parallel, which can naturally reduce the two errors through improving the three components of the upper bound. To strengthen this advantage, we encourage to cooperate these sub-networks by penalizing the difference of predictions made by their feature representations. With a fixed parameter budget, CoSCL can improve a variety of representative continual learning approaches by a large margin (e.g., up to 10.64% on CIFAR-100-SC, 9.33% on CIFAR-100-RS, 11.45% on CUB-200-2011 and 6.72% on Tiny-ImageNet) and achieve the new state-of-the-art performance.","url_abs":"https://arxiv.org/abs/2207.06543v1","url_pdf":"https://arxiv.org/pdf/2207.06543v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"coscl-cooperation-of-small-continual-learners","repo_url":"https://github.com/lywang3081/coscl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"coscl-cooperation-of-small-continual-learners","repo_url":"https://github.com/lywang3081/caf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"open-question","task_name":"Open-Ended Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2207.06543","atlas_url":"https://app.syntology.ai/?focus=2207.06543","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.06543"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lywang3081/coscl","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lywang3081/caf","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/lywang3081/CoSCL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"88d85883088e4691","entry":"gs_cal_mmoe","repo":"lywang3081/CoSCL","repo_kind":"official","path":"approaches/gs_coscl.py","file_url":"https://github.com/lywang3081/CoSCL/blob/HEAD/approaches/gs_coscl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88d85883088e4691"}},{"code_sha256_prefix":"d02101c3bf771dcb","entry":"Appr","repo":"lywang3081/CoSCL","repo_kind":"official","path":"approaches/gs_coscl.py","file_url":"https://github.com/lywang3081/CoSCL/blob/HEAD/approaches/gs_coscl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d02101c3bf771dcb"}},{"code_sha256_prefix":"cf3dfb506115c589","entry":"KLD","repo":"lywang3081/caf","repo_kind":"listed","path":"approaches/ppo_ewc_caf.py","file_url":"https://github.com/lywang3081/caf/blob/HEAD/approaches/ppo_ewc_caf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf3dfb506115c589"}},{"code_sha256_prefix":"98dc6385a4cc4799","entry":"PPO_EWC_CAF","repo":"lywang3081/caf","repo_kind":"listed","path":"approaches/ppo_ewc_caf.py","file_url":"https://github.com/lywang3081/caf/blob/HEAD/approaches/ppo_ewc_caf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98dc6385a4cc4799"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}