{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/continual-learning-forget-free-winning","title":"Continual Learning: Forget-free Winning Subnetworks for Video Representations","arxiv_id":"2312.11973","date":"2023-12-19","proceeding":null,"authors":["Haeyong Kang","Jaehong Yoon","Sung Ju Hwang","Chang D. Yoo"],"abstract":"Inspired by the Lottery Ticket Hypothesis (LTH), which highlights the existence of efficient subnetworks within larger, dense networks, a high-performing Winning Subnetwork (WSN) in terms of task performance under appropriate sparsity conditions is considered for various continual learning tasks. It leverages pre-existing weights from dense networks to achieve efficient learning in Task Incremental Learning (TIL) and Task-agnostic Incremental Learning (TaIL) scenarios. In Few-Shot Class Incremental Learning (FSCIL), a variation of WSN referred to as the Soft subnetwork (SoftNet) is designed to prevent overfitting when the data samples are scarce. Furthermore, the sparse reuse of WSN weights is considered for Video Incremental Learning (VIL). The use of Fourier Subneural Operator (FSO) within WSN is considered. It enables compact encoding of videos and identifies reusable subnetworks across varying bandwidths. We have integrated FSO into different architectural frameworks for continual learning, including VIL, TIL, and FSCIL. Our comprehensive experiments demonstrate FSO's effectiveness, significantly improving task performance at various convolutional representational levels. Specifically, FSO enhances higher-layer performance in TIL and FSCIL and lower-layer performance in VIL.","url_abs":"https://arxiv.org/abs/2312.11973v6","url_pdf":"https://arxiv.org/pdf/2312.11973v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"continual-learning-forget-free-winning","repo_url":"https://github.com/ihaeyong/pfnr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"continual-learning-forget-free-winning","repo_url":"https://github.com/ihaeyong/pnr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"few-shot-class-incremental-learning","task_name":"Few-Shot Class-Incremental Learning"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.11973","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.11973"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ihaeyong/pfnr","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ihaeyong/pnr","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":5,"ran_fixture":1,"ran_honours":1,"unverified":8},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1},"listed":{"samples":12,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"878fd412e7a78234","entry":"ActivationLayer","repo":"ihaeyong/pnr","repo_kind":"listed","path":"model_nerv.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/model_nerv.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"878fd412e7a78234"}},{"code_sha256_prefix":"4b4bad794e3e1cab","entry":"MLP","repo":"ihaeyong/pnr","repo_kind":"listed","path":"model_nerv.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/model_nerv.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b4bad794e3e1cab"}},{"code_sha256_prefix":"853ab1b19577355a","entry":"NormLayer","repo":"ihaeyong/pnr","repo_kind":"listed","path":"model_nerv.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/model_nerv.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"853ab1b19577355a"}},{"code_sha256_prefix":"e71c2a2bc2fc8909","entry":"compute_conv_output_size","repo":"ihaeyong/pnr","repo_kind":"listed","path":"model/subnet.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/model/subnet.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e71c2a2bc2fc8909"}},{"code_sha256_prefix":"c24fca3836221111","entry":"get_consolidated_masks","repo":"ihaeyong/pfnr","repo_kind":"official","path":"train_nerv.py","file_url":"https://github.com/ihaeyong/pfnr/blob/HEAD/train_nerv.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c24fca3836221111"}},{"code_sha256_prefix":"144b98b55047b2da","entry":"get_task_sparsity","repo":"ihaeyong/pfnr","repo_kind":"official","path":"train_nerv.py","file_url":"https://github.com/ihaeyong/pfnr/blob/HEAD/train_nerv.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"144b98b55047b2da"}},{"code_sha256_prefix":"913e2d426938c7d4","entry":"percentile","repo":"ihaeyong/pnr","repo_kind":"listed","path":"model/subnet.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/model/subnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"913e2d426938c7d4"}},{"code_sha256_prefix":"a1ba6b0c4d96716f","entry":"all_gather","repo":"ihaeyong/pnr","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1ba6b0c4d96716f"}},{"code_sha256_prefix":"439067335e7b67b5","entry":"all_reduce","repo":"ihaeyong/pnr","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"439067335e7b67b5"}},{"code_sha256_prefix":"d8684b8a9ca3ef7a","entry":"evaluate","repo":"ihaeyong/pfnr","repo_kind":"official","path":"train_nerv.py","file_url":"https://github.com/ihaeyong/pfnr/blob/HEAD/train_nerv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d8684b8a9ca3ef7a"}},{"code_sha256_prefix":"cc46e27629d64950","entry":"get_consolidated_masks","repo":"ihaeyong/pnr","repo_kind":"listed","path":"plot_nerv_eval.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/plot_nerv_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc46e27629d64950"}},{"code_sha256_prefix":"f0e8e55d92917b22","entry":"get_coused_sparsity","repo":"ihaeyong/pnr","repo_kind":"listed","path":"train_nerv_eval.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/train_nerv_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0e8e55d92917b22"}},{"code_sha256_prefix":"49fb8932e40b133e","entry":"get_reused_sparsity","repo":"ihaeyong/pnr","repo_kind":"listed","path":"train_nerv_eval.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/train_nerv_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49fb8932e40b133e"}},{"code_sha256_prefix":"000804ce4ed442ee","entry":"get_task_sparsity","repo":"ihaeyong/pnr","repo_kind":"listed","path":"train_nerv_eval.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/train_nerv_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"000804ce4ed442ee"}},{"code_sha256_prefix":"9ba78d582d59f8ed","entry":"quantize_per_tensor","repo":"ihaeyong/pnr","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/ihaeyong/pnr/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ba78d582d59f8ed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}