{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mimicking-the-oracle-an-initial-phase","title":"Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning","arxiv_id":"2112.04731","date":"2021-12-09","proceeding":"CVPR 2022 1","authors":["Yujun Shi","Kuangqi Zhou","Jian Liang","Zihang Jiang","Jiashi Feng","Philip Torr","Song Bai","Vincent Y. F. Tan"],"abstract":"Class Incremental Learning (CIL) aims at learning a multi-class classifier in a phase-by-phase manner, in which only data of a subset of the classes are provided at each phase. Previous works mainly focus on mitigating forgetting in phases after the initial one. However, we find that improving CIL at its initial phase is also a promising direction. Specifically, we experimentally show that directly encouraging CIL Learner at the initial phase to output similar representations as the model jointly trained on all classes can greatly boost the CIL performance. Motivated by this, we study the difference between a na\\\"ively-trained initial-phase model and the oracle model. Specifically, since one major difference between these two models is the number of training classes, we investigate how such difference affects the model representations. We find that, with fewer training classes, the data representations of each class lie in a long and narrow region; with more training classes, the representations of each class scatter more uniformly. Inspired by this observation, we propose Class-wise Decorrelation (CwD) that effectively regularizes representations of each class to scatter more uniformly, thus mimicking the model jointly trained with all classes (i.e., the oracle model). Our CwD is simple to implement and easy to plug into existing methods. Extensive experiments on various benchmark datasets show that CwD consistently and significantly improves the performance of existing state-of-the-art methods by around 1\\% to 3\\%. Code will be released.","url_abs":"https://arxiv.org/abs/2112.04731v5","url_pdf":"https://arxiv.org/pdf/2112.04731v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mimicking-the-oracle-an-initial-phase","repo_url":"https://github.com/yujun-shi/cwd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.04731","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.04731"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yujun-shi/cwd","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_fixture":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9b12f69ea3486b68","entry":"compute_confusion_matrix","repo":"yujun-shi/cwd","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/yujun-shi/cwd/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b12f69ea3486b68"}},{"code_sha256_prefix":"2e179edce79cd470","entry":"last_layer_analysis","repo":"yujun-shi/cwd","repo_kind":"official","path":"src/last_layer_analysis.py","file_url":"https://github.com/yujun-shi/cwd/blob/HEAD/src/last_layer_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e179edce79cd470"}},{"code_sha256_prefix":"3e30d88eaef01190","entry":"off_diagonal","repo":"yujun-shi/cwd","repo_kind":"official","path":"src/approach/aux_loss.py","file_url":"https://github.com/yujun-shi/cwd/blob/HEAD/src/approach/aux_loss.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e30d88eaef01190"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}