{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scale-online-self-supervised-lifelong","title":"SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge","arxiv_id":"2208.11266","date":"2022-08-24","proceeding":null,"authors":["Xiaofan Yu","Yunhui Guo","Sicun Gao","Tajana Rosing"],"abstract":"Unsupervised lifelong learning refers to the ability to learn over time while memorizing previous patterns without supervision. Although great progress has been made in this direction, existing work often assumes strong prior knowledge about the incoming data (e.g., knowing the class boundaries), which can be impossible to obtain in complex and unpredictable environments. In this paper, motivated by real-world scenarios, we propose a more practical problem setting called online self-supervised lifelong learning without prior knowledge. The proposed setting is challenging due to the non-iid and single-pass data, the absence of external supervision, and no prior knowledge. To address the challenges, we propose Self-Supervised ContrAstive Lifelong LEarning without Prior Knowledge (SCALE) which can extract and memorize representations on the fly purely from the data continuum. SCALE is designed around three major components: a pseudo-supervised contrastive loss, a self-supervised forgetting loss, and an online memory update for uniform subset selection. All three components are designed to work collaboratively to maximize learning performance. We perform comprehensive experiments of SCALE under iid and four non-iid data streams. The results show that SCALE outperforms the state-of-the-art algorithm in all settings with improvements up to 3.83%, 2.77% and 5.86% in terms of kNN accuracy on CIFAR-10, CIFAR-100, and TinyImageNet datasets.","url_abs":"https://arxiv.org/abs/2208.11266v5","url_pdf":"https://arxiv.org/pdf/2208.11266v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scale-online-self-supervised-lifelong","repo_url":"https://github.com/orienfish/scale","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"lifelong-learning","task_name":"Lifelong learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2208.11266","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2208.11266"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/orienfish/scale","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":4,"unverified":10},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b0f936d4d6ae3b8c","entry":"accuracy","repo":"orienfish/scale","repo_kind":"official","path":"OnlineContrast/util.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/OnlineContrast/util.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b0f936d4d6ae3b8c"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"orienfish/scale","repo_kind":"official","path":"networks/resnet_pnn.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/networks/resnet_pnn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"2a80220dabcb742a","entry":"conv1x1","repo":"orienfish/scale","repo_kind":"official","path":"networks/resnet_big.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/networks/resnet_big.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a80220dabcb742a"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"orienfish/scale","repo_kind":"official","path":"networks/resnet_pnn.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/networks/resnet_pnn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"600ff2c45e0de056","entry":"conv3x3","repo":"orienfish/scale","repo_kind":"official","path":"networks/resnet_big.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/networks/resnet_big.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"600ff2c45e0de056"}},{"code_sha256_prefix":"617ac050b1b5d75f","entry":"PairEnum","repo":"orienfish/scale","repo_kind":"official","path":"OnlineContrast/losses/supcon.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/OnlineContrast/losses/supcon.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"617ac050b1b5d75f"}},{"code_sha256_prefix":"c3e5242b26d55633","entry":"adjust_learning_rate","repo":"orienfish/scale","repo_kind":"official","path":"set_utils.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/set_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3e5242b26d55633"}},{"code_sha256_prefix":"60bc24fbf225a7b6","entry":"eval_acc","repo":"orienfish/scale","repo_kind":"official","path":"eval_utils.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60bc24fbf225a7b6"}},{"code_sha256_prefix":"0e4c29b9efff1c29","entry":"eval_knn","repo":"orienfish/scale","repo_kind":"official","path":"eval_utils.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e4c29b9efff1c29"}},{"code_sha256_prefix":"99764303673acdd0","entry":"reservoir","repo":"orienfish/scale","repo_kind":"official","path":"OnlineContrast/memory.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/OnlineContrast/memory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99764303673acdd0"}},{"code_sha256_prefix":"f2039bb8839fe151","entry":"resnet18_pnn","repo":"orienfish/scale","repo_kind":"official","path":"networks/resnet_pnn.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/networks/resnet_pnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2039bb8839fe151"}},{"code_sha256_prefix":"7b29daa3d6334d6f","entry":"similarity_mask_new","repo":"orienfish/scale","repo_kind":"official","path":"OnlineContrast/losses/supcon.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/OnlineContrast/losses/supcon.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b29daa3d6334d6f"}},{"code_sha256_prefix":"d52134cc529e8c56","entry":"similarity_mask_old","repo":"orienfish/scale","repo_kind":"official","path":"OnlineContrast/losses/supcon.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/OnlineContrast/losses/supcon.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d52134cc529e8c56"}},{"code_sha256_prefix":"9020d53a124c5931","entry":"sparse2coarse","repo":"orienfish/scale","repo_kind":"official","path":"data_utils.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9020d53a124c5931"}},{"code_sha256_prefix":"875fdfa37e3fdeb1","entry":"tsne_simil","repo":"orienfish/scale","repo_kind":"official","path":"eval_utils.py","file_url":"https://github.com/orienfish/scale/blob/HEAD/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"875fdfa37e3fdeb1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}