{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-03473","title":"Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Experts","arxiv_id":"2602.03473","date":"2026-02-03","proceeding":"ICML","authors":["Meng Lou","Yunxiang Fu","Yizhou Yu"],"abstract":"Continual learning, especially class-incremental learning (CIL), on the basis of a pre-trained model (PTM) has garnered substantial research interest in recent years. However, how to effectively learn both discriminative and comprehensive feature representations while maintaining stability and plasticity over very long task sequences remains an open problem. We propose CaRE, a scalable {C}ontinual Le{a}rner with efficient Bi-Level {R}outing Mixture-of-{E}xperts (BR-MoE). The core idea of BR-MoE is a bi-level routing mechanism: a router selection stage that dynamically activates relevant task-specific routers, followed by an expert routing phase that dynamically activates and aggregates experts, aiming to inject discriminative and comprehensive representations into every intermediate network layer. On the other hand, we introduce a challenging dataset, OmniBenchmark-1K, for CIL performance evaluation on very long task sequences with hundreds of tasks. Extensive experiments show that CaRE demonstrates leading performance across a variety of datasets and task settings, including commonly used CIL datasets with classical CIL settings (e.g., 5-20 tasks). To the best of our knowledge, CaRE is the first continual learner that scales to very long task sequences (ranging from 100 to over 300 non-overlapping tasks), while outperforming all baselines by a large margin on such task sequences. We hope that this work will inspire further research into continual learning over extremely long task sequences. Code and dataset are publicly released at https://github.com/LMMMEng/CaRE.","url_abs":"https://arxiv.org/abs/2602.03473","url_pdf":"https://arxiv.org/pdf/2602.03473","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2602.03473","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.03473"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/LMMMEng/CaRE","reach":null}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7b7f101ebda35088","entry":"Adapter","repo":"LMMMEng/CaRE","repo_kind":"found_in_text","path":"backbone/vit_brmoe.py","file_url":"https://github.com/LMMMEng/CaRE/blob/HEAD/backbone/vit_brmoe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b7f101ebda35088"}},{"code_sha256_prefix":"52a9b5305ff1f5ef","entry":"Attention","repo":"LMMMEng/CaRE","repo_kind":"found_in_text","path":"backbone/vit_brmoe.py","file_url":"https://github.com/LMMMEng/CaRE/blob/HEAD/backbone/vit_brmoe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"52a9b5305ff1f5ef"}},{"code_sha256_prefix":"e5a863df7a76f7e2","entry":"Block","repo":"LMMMEng/CaRE","repo_kind":"found_in_text","path":"backbone/vit_brmoe.py","file_url":"https://github.com/LMMMEng/CaRE/blob/HEAD/backbone/vit_brmoe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5a863df7a76f7e2"}},{"code_sha256_prefix":"682e645892e64729","entry":"MoEAdapter","repo":"LMMMEng/CaRE","repo_kind":"found_in_text","path":"backbone/vit_brmoe.py","file_url":"https://github.com/LMMMEng/CaRE/blob/HEAD/backbone/vit_brmoe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"682e645892e64729"}},{"code_sha256_prefix":"afa43a669cda3d4f","entry":"VisionTransformer","repo":"LMMMEng/CaRE","repo_kind":"found_in_text","path":"backbone/vit_brmoe.py","file_url":"https://github.com/LMMMEng/CaRE/blob/HEAD/backbone/vit_brmoe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"afa43a669cda3d4f"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}