{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/plasticity-optimized-complementary-networks","title":"Plasticity-Optimized Complementary Networks for Unsupervised Continual Learning","arxiv_id":"2309.06086","date":"2023-09-12","proceeding":null,"authors":["Alex Gomez-Villa","Bartlomiej Twardowski","Kai Wang","Joost Van de Weijer"],"abstract":"Continuous unsupervised representation learning (CURL) research has greatly benefited from improvements in self-supervised learning (SSL) techniques. As a result, existing CURL methods using SSL can learn high-quality representations without any labels, but with a notable performance drop when learning on a many-tasks data stream. We hypothesize that this is caused by the regularization losses that are imposed to prevent forgetting, leading to a suboptimal plasticity-stability trade-off: they either do not adapt fully to the incoming data (low plasticity), or incur significant forgetting when allowed to fully adapt to a new SSL pretext-task (low stability). In this work, we propose to train an expert network that is relieved of the duty of keeping the previous knowledge and can focus on performing optimally on the new tasks (optimizing plasticity). In the second phase, we combine this new knowledge with the previous network in an adaptation-retrospection phase to avoid forgetting and initialize a new expert with the knowledge of the old network. We perform several experiments showing that our proposed approach outperforms other CURL exemplar-free methods in few- and many-task split settings. Furthermore, we show how to adapt our approach to semi-supervised continual learning (Semi-SCL) and show that we surpass the accuracy of other exemplar-free Semi-SCL methods and reach the results of some others that use exemplars.","url_abs":"https://arxiv.org/abs/2309.06086v1","url_pdf":"https://arxiv.org/pdf/2309.06086v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"plasticity-optimized-complementary-networks","repo_url":"https://github.com/alviur/pocon_wacv2024","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"exemplar-free","task_name":"Exemplar-Free"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"unsupervised-class-incremental-learning","task_name":"unsupervised class-incremental learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.06086","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.06086"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alviur/pocon_wacv2024","reach":{"status":"ok"}}],"summary":{"ran":5},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"ebb1a59bc40babbb","entry":"align_loss","repo":"alviur/pocon_wacv2024","repo_kind":"official","path":"src/align_uniform_loss.py","file_url":"https://github.com/alviur/pocon_wacv2024/blob/HEAD/src/align_uniform_loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ebb1a59bc40babbb"}},{"code_sha256_prefix":"e2a5fdc0a059abac","entry":"conv_block","repo":"alviur/pocon_wacv2024","repo_kind":"official","path":"src/approach/FT_online.py","file_url":"https://github.com/alviur/pocon_wacv2024/blob/HEAD/src/approach/FT_online.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e2a5fdc0a059abac"}},{"code_sha256_prefix":"1007009ff519b620","entry":"corrLoss","repo":"alviur/pocon_wacv2024","repo_kind":"official","path":"src/approach/FT_online.py","file_url":"https://github.com/alviur/pocon_wacv2024/blob/HEAD/src/approach/FT_online.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1007009ff519b620"}},{"code_sha256_prefix":"ca17f953a7c13ff5","entry":"str2alg","repo":"alviur/pocon_wacv2024","repo_kind":"official","path":"src/imageNet_resizer.py","file_url":"https://github.com/alviur/pocon_wacv2024/blob/HEAD/src/imageNet_resizer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ca17f953a7c13ff5"}},{"code_sha256_prefix":"8d3d5f756818e024","entry":"uniform_loss","repo":"alviur/pocon_wacv2024","repo_kind":"official","path":"src/align_uniform_loss.py","file_url":"https://github.com/alviur/pocon_wacv2024/blob/HEAD/src/align_uniform_loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d3d5f756818e024"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}