{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stabilizing-contrastive-rl-techniques-for","title":"Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data","arxiv_id":"2306.03346","date":"2023-06-06","proceeding":null,"authors":["Chongyi Zheng","Benjamin Eysenbach","Homer Walke","Patrick Yin","Kuan Fang","Ruslan Salakhutdinov","Sergey Levine"],"abstract":"Robotic systems that rely primarily on self-supervised learning have the potential to decrease the amount of human annotation and engineering effort required to learn control strategies. In the same way that prior robotic systems have leveraged self-supervised techniques from computer vision (CV) and natural language processing (NLP), our work builds on prior work showing that the reinforcement learning (RL) itself can be cast as a self-supervised problem: learning to reach any goal without human-specified rewards or labels. Despite the seeming appeal, little (if any) prior work has demonstrated how self-supervised RL methods can be practically deployed on robotic systems. By first studying a challenging simulated version of this task, we discover design decisions about architectures and hyperparameters that increase the success rate by $2 \\times$. These findings lay the groundwork for our main result: we demonstrate that a self-supervised RL algorithm based on contrastive learning can solve real-world, image-based robotic manipulation tasks, with tasks being specified by a single goal image provided after training.","url_abs":"https://arxiv.org/abs/2306.03346v2","url_pdf":"https://arxiv.org/pdf/2306.03346v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stabilizing-contrastive-rl-techniques-for","repo_url":"https://github.com/chongyi-zheng/stable_contrastive_rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.03346","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.03346"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chongyi-zheng/stable_contrastive_rl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fadf5f762ff04cd6","entry":"create_stats_ordered_dict","repo":"chongyi-zheng/stable_contrastive_rl","repo_kind":"official","path":"rlkit/core/eval_util.py","file_url":"https://github.com/chongyi-zheng/stable_contrastive_rl/blob/HEAD/rlkit/core/eval_util.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fadf5f762ff04cd6"}},{"code_sha256_prefix":"a5d10a0b704c8db7","entry":"get_average_returns","repo":"chongyi-zheng/stable_contrastive_rl","repo_kind":"official","path":"rlkit/core/eval_util.py","file_url":"https://github.com/chongyi-zheng/stable_contrastive_rl/blob/HEAD/rlkit/core/eval_util.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5d10a0b704c8db7"}},{"code_sha256_prefix":"6dc14f1205c31904","entry":"add_prefix","repo":"chongyi-zheng/stable_contrastive_rl","repo_kind":"official","path":"rlkit/core/logging.py","file_url":"https://github.com/chongyi-zheng/stable_contrastive_rl/blob/HEAD/rlkit/core/logging.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6dc14f1205c31904"}},{"code_sha256_prefix":"9817c2d99db83a09","entry":"append_log","repo":"chongyi-zheng/stable_contrastive_rl","repo_kind":"official","path":"rlkit/core/logging.py","file_url":"https://github.com/chongyi-zheng/stable_contrastive_rl/blob/HEAD/rlkit/core/logging.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9817c2d99db83a09"}},{"code_sha256_prefix":"26a21a70206f48ab","entry":"reopen","repo":"chongyi-zheng/stable_contrastive_rl","repo_kind":"official","path":"rlkit/core/logging.py","file_url":"https://github.com/chongyi-zheng/stable_contrastive_rl/blob/HEAD/rlkit/core/logging.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26a21a70206f48ab"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}