{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/does-self-supervised-learning-really-improve","title":"Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?","arxiv_id":"2206.05266","date":"2022-06-10","proceeding":null,"authors":["Xiang Li","Jinghuan Shang","Srijan Das","Michael S. Ryoo"],"abstract":"We investigate whether self-supervised learning (SSL) can improve online reinforcement learning (RL) from pixels. We extend the contrastive reinforcement learning framework (e.g., CURL) that jointly optimizes SSL and RL losses and conduct an extensive amount of experiments with various self-supervised losses. Our observations suggest that the existing SSL framework for RL fails to bring meaningful improvement over the baselines only taking advantage of image augmentation when the same amount of data and augmentation is used. We further perform evolutionary searches to find the optimal combination of multiple self-supervised losses for RL, but find that even such a loss combination fails to meaningfully outperform the methods that only utilize carefully designed image augmentations. After evaluating these approaches together in multiple different environments including a real-world robot environment, we confirm that no single self-supervised loss or image augmentation method can dominate all environments and that the current framework for joint optimization of SSL and RL is limited. Finally, we conduct the ablation study on multiple factors and demonstrate the properties of representations learned with different approaches.","url_abs":"https://arxiv.org/abs/2206.05266v4","url_pdf":"https://arxiv.org/pdf/2206.05266v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"does-self-supervised-learning-really-improve","repo_url":"https://github.com/LostXine/elo-sac","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"does-self-supervised-learning-really-improve","repo_url":"https://github.com/lostxine/elo-rainbow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-augmentation","task_name":"Image Augmentation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.05266","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.05266"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/LostXine/elo-sac","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/LostXine/elo-rainbow","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lostxine/elo-rainbow","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"daa7b3a0355eb7c3","entry":"gaussian_logprob","repo":"LostXine/elo-sac","repo_kind":"official","path":"elo_sac.py","file_url":"https://github.com/LostXine/elo-sac/blob/HEAD/elo_sac.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"daa7b3a0355eb7c3"}},{"code_sha256_prefix":"42958997018c3581","entry":"make_dir","repo":"LostXine/elo-sac","repo_kind":"official","path":"utils.py","file_url":"https://github.com/LostXine/elo-sac/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42958997018c3581"}},{"code_sha256_prefix":"ef505603f4520121","entry":"make_handler","repo":"LostXine/elo-rainbow","repo_kind":"official","path":"server.py","file_url":"https://github.com/LostXine/elo-rainbow/blob/HEAD/server.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef505603f4520121"}},{"code_sha256_prefix":"63cc0b17d508c362","entry":"module_hash","repo":"LostXine/elo-sac","repo_kind":"official","path":"utils.py","file_url":"https://github.com/LostXine/elo-sac/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63cc0b17d508c362"}},{"code_sha256_prefix":"e4f442d1770eacd1","entry":"squash","repo":"LostXine/elo-sac","repo_kind":"official","path":"elo_sac.py","file_url":"https://github.com/LostXine/elo-sac/blob/HEAD/elo_sac.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4f442d1770eacd1"}},{"code_sha256_prefix":"ced159fb4f2d853b","entry":"make_encoder","repo":"LostXine/elo-sac","repo_kind":"official","path":"encoder.py","file_url":"https://github.com/LostXine/elo-sac/blob/HEAD/encoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ced159fb4f2d853b"}},{"code_sha256_prefix":"6428d2b9919f888a","entry":"preprocess_obs","repo":"LostXine/elo-sac","repo_kind":"official","path":"utils.py","file_url":"https://github.com/LostXine/elo-sac/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6428d2b9919f888a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}