{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distilling-causal-effect-of-data-in-class","title":"Distilling Causal Effect of Data in Class-Incremental Learning","arxiv_id":"2103.01737","date":"2021-03-02","proceeding":"CVPR 2021 1","authors":["Xinting Hu","Kaihua Tang","Chunyan Miao","Xian-Sheng Hua","Hanwang Zhang"],"abstract":"We propose a causal framework to explain the catastrophic forgetting in Class-Incremental Learning (CIL) and then derive a novel distillation method that is orthogonal to the existing anti-forgetting techniques, such as data replay and feature/label distillation. We first 1) place CIL into the framework, 2) answer why the forgetting happens: the causal effect of the old data is lost in new training, and then 3) explain how the existing techniques mitigate it: they bring the causal effect back. Based on the framework, we find that although the feature/label distillation is storage-efficient, its causal effect is not coherent with the end-to-end feature learning merit, which is however preserved by data replay. To this end, we propose to distill the Colliding Effect between the old and the new data, which is fundamentally equivalent to the causal effect of data replay, but without any cost of replay storage. Thanks to the causal effect analysis, we can further capture the Incremental Momentum Effect of the data stream, removing which can help to retain the old effect overwhelmed by the new data effect, and thus alleviate the forgetting of the old class in testing. Extensive experiments on three CIL benchmarks: CIFAR-100, ImageNet-Sub&Full, show that the proposed causal effect distillation can improve various state-of-the-art CIL methods by a large margin (0.72%--9.06%).","url_abs":"https://arxiv.org/abs/2103.01737v3","url_pdf":"https://arxiv.org/pdf/2103.01737v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distilling-causal-effect-of-data-in-class","repo_url":"https://github.com/JoyHuYY1412/DDE_CIL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2103.01737","atlas_url":"https://app.syntology.ai/?focus=2103.01737","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.01737"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JoyHuYY1412/DDE_CIL","reach":null}],"summary":{"ran_honours":2,"ran_fixture":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"49e14a6aa465a8ed","entry":"compute_features_FeatCluster","repo":"JoyHuYY1412/DDE_CIL","repo_kind":"official","path":"utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","file_url":"https://github.com/JoyHuYY1412/DDE_CIL/blob/HEAD/utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49e14a6aa465a8ed"}},{"code_sha256_prefix":"f72a592dc903649f","entry":"compute_features_FeatCluster_imagenet","repo":"JoyHuYY1412/DDE_CIL","repo_kind":"official","path":"utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","file_url":"https://github.com/JoyHuYY1412/DDE_CIL/blob/HEAD/utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f72a592dc903649f"}},{"code_sha256_prefix":"f47612df435b1ff0","entry":"incremental_train_and_eval_DCE_MR_LF_TDE","repo":"JoyHuYY1412/DDE_CIL","repo_kind":"official","path":"utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","file_url":"https://github.com/JoyHuYY1412/DDE_CIL/blob/HEAD/utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f47612df435b1ff0"}},{"code_sha256_prefix":"74b4050dbd108caa","entry":"pdist","repo":"JoyHuYY1412/DDE_CIL","repo_kind":"official","path":"utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","file_url":"https://github.com/JoyHuYY1412/DDE_CIL/blob/HEAD/utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"74b4050dbd108caa"}},{"code_sha256_prefix":"34d5b20ed9572a5e","entry":"get_new_scores_before_scale","repo":"JoyHuYY1412/DDE_CIL","repo_kind":"official","path":"utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","file_url":"https://github.com/JoyHuYY1412/DDE_CIL/blob/HEAD/utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34d5b20ed9572a5e"}},{"code_sha256_prefix":"caad535664fadc14","entry":"get_old_scores_before_scale","repo":"JoyHuYY1412/DDE_CIL","repo_kind":"official","path":"utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","file_url":"https://github.com/JoyHuYY1412/DDE_CIL/blob/HEAD/utils_incremental/incremental_train_and_eval_DCE_MR_LF_TDE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"caad535664fadc14"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}