{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/demonstration-free-autonomous-reinforcement","title":"Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum","arxiv_id":"2305.09943","date":"2023-05-17","proceeding":null,"authors":["Jigang Kim","Daesol Cho","H. Jin Kim"],"abstract":"While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.","url_abs":"https://arxiv.org/abs/2305.09943v2","url_pdf":"https://arxiv.org/pdf/2305.09943v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"demonstration-free-autonomous-reinforcement","repo_url":"https://github.com/snu-larr/ibc_official","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.09943","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.09943"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/snu-larr/ibc_official","reach":null}],"summary":{"ran_fixture":2,"ran_honours":2,"ran_draft_wrong":1,"ran_violates":1,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"25e3e1ced355d6ba","entry":"add_noise_to_goal","repo":"snu-larr/ibc_official","repo_kind":"official","path":"ibc.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/ibc.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25e3e1ced355d6ba"}},{"code_sha256_prefix":"6714a60d5440418a","entry":"c_double","repo":"snu-larr/ibc_official","repo_kind":"official","path":"hgg/hgg.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/hgg/hgg.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6714a60d5440418a"}},{"code_sha256_prefix":"c23ba227c152282e","entry":"gcc_complie","repo":"snu-larr/ibc_official","repo_kind":"official","path":"hgg/hgg.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/hgg/hgg.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c23ba227c152282e"}},{"code_sha256_prefix":"280c5717797f3baa","entry":"goal_concat","repo":"snu-larr/ibc_official","repo_kind":"official","path":"hgg/hgg.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/hgg/hgg.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"280c5717797f3baa"}},{"code_sha256_prefix":"b02202bf22dda2af","entry":"goal_distance","repo":"snu-larr/ibc_official","repo_kind":"official","path":"hgg/hgg.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/hgg/hgg.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b02202bf22dda2af"}},{"code_sha256_prefix":"13ef231a67b07fe5","entry":"normalize_obs","repo":"snu-larr/ibc_official","repo_kind":"official","path":"ibc.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/ibc.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"13ef231a67b07fe5"}},{"code_sha256_prefix":"e52b233a9f359b04","entry":"MatchSampler","repo":"snu-larr/ibc_official","repo_kind":"official","path":"hgg/hgg.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/hgg/hgg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e52b233a9f359b04"}},{"code_sha256_prefix":"92d0357f607b064c","entry":"gcc_load_lib","repo":"snu-larr/ibc_official","repo_kind":"official","path":"hgg/hgg.py","file_url":"https://github.com/snu-larr/ibc_official/blob/HEAD/hgg/hgg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"92d0357f607b064c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}