{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revisiting-supervision-for-continual","title":"Revisiting Supervision for Continual Representation Learning","arxiv_id":"2311.13321","date":"2023-11-22","proceeding":null,"authors":["Daniel Marczak","Sebastian Cygert","Tomasz Trzciński","Bartłomiej Twardowski"],"abstract":"In the field of continual learning, models are designed to learn tasks one after the other. While most research has centered on supervised continual learning, there is a growing interest in unsupervised continual learning, which makes use of the vast amounts of unlabeled data. Recent studies have highlighted the strengths of unsupervised methods, particularly self-supervised learning, in providing robust representations. The improved transferability of those representations built with self-supervised methods is often associated with the role played by the multi-layer perceptron projector. In this work, we depart from this observation and reexamine the role of supervision in continual representation learning. We reckon that additional information, such as human annotations, should not deteriorate the quality of representations. Our findings show that supervised models when enhanced with a multi-layer perceptron head, can outperform self-supervised models in continual representation learning. This highlights the importance of the multi-layer perceptron projector in shaping feature transferability across a sequence of tasks in continual learning. The code is available on github: https://github.com/danielm1405/sl-vs-ssl-cl.","url_abs":"https://arxiv.org/abs/2311.13321v2","url_pdf":"https://arxiv.org/pdf/2311.13321v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revisiting-supervision-for-continual","repo_url":"https://github.com/danielm1405/sl-vs-ssl-cl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.13321","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.13321"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/danielm1405/sl-vs-ssl-cl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":10,"unverified":2},"by_repo_kind":{"official":{"samples":12,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"132a949a804eb81e","entry":"barlow_loss_func","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"cassle/losses/barlow.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/cassle/losses/barlow.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"132a949a804eb81e"}},{"code_sha256_prefix":"33aef221912d8415","entry":"byol_loss_func","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"cassle/losses/byol.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/cassle/losses/byol.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33aef221912d8415"}},{"code_sha256_prefix":"98c308caf3b29f19","entry":"calc_prototypes","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"nmc-classifier.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/nmc-classifier.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98c308caf3b29f19"}},{"code_sha256_prefix":"2e8bb65a92ffe11e","entry":"dict_to_list","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"main_continual.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/main_continual.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e8bb65a92ffe11e"}},{"code_sha256_prefix":"f7739772dfd565b5","entry":"directory_find","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"main_eval_transfer_knn.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/main_eval_transfer_knn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7739772dfd565b5"}},{"code_sha256_prefix":"2c0a957445944395","entry":"extract_features","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"main_eval_transfer_knn.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/main_eval_transfer_knn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c0a957445944395"}},{"code_sha256_prefix":"71100c26c7ffe43b","entry":"extract_features","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"main_knn.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/main_knn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71100c26c7ffe43b"}},{"code_sha256_prefix":"26e60270b1fdfcfe","entry":"get_initialized_backbone","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"main_eval_transfer_knn.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/main_eval_transfer_knn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26e60270b1fdfcfe"}},{"code_sha256_prefix":"2b245d2b8c6563d6","entry":"get_per_class_embeddings_and_classes","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"nmc-classifier.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/nmc-classifier.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b245d2b8c6563d6"}},{"code_sha256_prefix":"0cc8b473b7806887","entry":"str_to_dict","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"main_continual.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/main_continual.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0cc8b473b7806887"}},{"code_sha256_prefix":"ce432f2e9653c7a3","entry":"deepclusterv2_loss_func","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"cassle/losses/deepclusterv2.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/cassle/losses/deepclusterv2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce432f2e9653c7a3"}},{"code_sha256_prefix":"3fbe4ea1364d66b8","entry":"get_initialized_backbone","repo":"danielm1405/sl-vs-ssl-cl","repo_kind":"official","path":"nmc-classifier.py","file_url":"https://github.com/danielm1405/sl-vs-ssl-cl/blob/HEAD/nmc-classifier.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fbe4ea1364d66b8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}