{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/behavior-contrastive-learning-for","title":"Behavior Contrastive Learning for Unsupervised Skill Discovery","arxiv_id":"2305.04477","date":"2023-05-08","proceeding":null,"authors":["Rushuai Yang","Chenjia Bai","Hongyi Guo","Siyuan Li","Bin Zhao","Zhen Wang","Peng Liu","Xuelong Li"],"abstract":"In reinforcement learning, unsupervised skill discovery aims to learn diverse skills without extrinsic rewards. Previous methods discover skills by maximizing the mutual information (MI) between states and skills. However, such an MI objective tends to learn simple and static skills and may hinder exploration. In this paper, we propose a novel unsupervised skill discovery method through contrastive learning among behaviors, which makes the agent produce similar behaviors for the same skill and diverse behaviors for different skills. Under mild assumptions, our objective maximizes the MI between different behaviors based on the same skill, which serves as an upper bound of the previous MI objective. Meanwhile, our method implicitly increases the state entropy to obtain better state coverage. We evaluate our method on challenging mazes and continuous control tasks. The results show that our method generates diverse and far-reaching skills, and also obtains competitive performance in downstream tasks compared to the state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2305.04477v1","url_pdf":"https://arxiv.org/pdf/2305.04477v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"behavior-contrastive-learning-for","repo_url":"https://github.com/rooshy-yang/becl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"continuous-control","task_name":"continuous-control"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.04477","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.04477"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/rll-research/cic","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Rooshy-yang/BeCL","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rooshy-yang/becl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"53554f444b3a77db","entry":"BECL","repo":"rooshy-yang/becl","repo_kind":"official","path":"agent/becl.py","file_url":"https://github.com/rooshy-yang/becl/blob/HEAD/agent/becl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53554f444b3a77db"}},{"code_sha256_prefix":"9f840a22d31a89bc","entry":"episode_len","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"replay_buffer.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/replay_buffer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f840a22d31a89bc"}},{"code_sha256_prefix":"2478bb95b38ef24f","entry":"grad_norm","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"utils.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2478bb95b38ef24f"}},{"code_sha256_prefix":"184876e5f4ab6562","entry":"load_episode","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"replay_buffer.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/replay_buffer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"184876e5f4ab6562"}},{"code_sha256_prefix":"717d02cf517c8633","entry":"make_replay_loader","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"replay_buffer.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/replay_buffer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"717d02cf517c8633"}},{"code_sha256_prefix":"8708aaca52006e47","entry":"param_norm","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"utils.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8708aaca52006e47"}},{"code_sha256_prefix":"6244e9922f4e0610","entry":"to_torch","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"utils.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"6244e9922f4e0610"}},{"code_sha256_prefix":"2a76dfe317adeb03","entry":"make_agent","repo":"Rooshy-yang/BeCL","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/Rooshy-yang/BeCL/blob/HEAD/finetune.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a76dfe317adeb03"}},{"code_sha256_prefix":"2d7fab6117dbc6a8","entry":"weight_init","repo":"rooshy-yang/becl","repo_kind":"official","path":"agent/becl.py","file_url":"https://github.com/rooshy-yang/becl/blob/HEAD/agent/becl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d7fab6117dbc6a8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}