{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-from-guided-play-a-scheduled","title":"Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning","arxiv_id":"2112.08932","date":"2021-12-16","proceeding":null,"authors":["Trevor Ablett","Bryan Chan","Jonathan Kelly"],"abstract":"Effective exploration continues to be a significant challenge that prevents the deployment of reinforcement learning for many physical systems. This is particularly true for systems with continuous and high-dimensional state and action spaces, such as robotic manipulators. The challenge is accentuated in the sparse rewards setting, where the low-level state information required for the design of dense rewards is unavailable. Adversarial imitation learning (AIL) can partially overcome this barrier by leveraging expert-generated demonstrations of optimal behaviour and providing, essentially, a replacement for dense reward information. Unfortunately, the availability of expert demonstrations does not necessarily improve an agent's capability to explore effectively and, as we empirically show, can lead to inefficient or stagnated learning. We present Learning from Guided Play (LfGP), a framework in which we leverage expert demonstrations of, in addition to a main task, multiple auxiliary tasks. Subsequently, a hierarchical model is used to learn each task reward and policy through a modified AIL procedure, in which exploration of all tasks is enforced via a scheduler composing different tasks together. This affords many benefits: learning efficiency is improved for main tasks with challenging bottleneck transitions, expert data becomes reusable between tasks, and transfer learning through the reuse of learned auxiliary task models becomes possible. Our experimental results in a challenging multitask robotic manipulation domain indicate that our method compares favourably to supervised imitation learning and to a state-of-the-art AIL method. Code is available at https://github.com/utiasSTARS/lfgp.","url_abs":"https://arxiv.org/abs/2112.08932v3","url_pdf":"https://arxiv.org/pdf/2112.08932v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-from-guided-play-a-scheduled","repo_url":"https://github.com/utiasstars/lfgp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[{"slug":"lfgp-data","name":"LfGP Data","full_name":"Learning from Guided Play Expert Data and Trained Models"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2112.08932","atlas_url":"https://app.syntology.ai/?focus=2112.08932","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.08932"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/utiasstars/lfgp","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9e1d0b6acb03b7a5","entry":"HierarchicalRLAgent","repo":"utiasstars/lfgp","repo_kind":"official","path":"rl_sandbox/rl_sandbox/agents/hrl_agents.py","file_url":"https://github.com/utiasstars/lfgp/blob/HEAD/rl_sandbox/rl_sandbox/agents/hrl_agents.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e1d0b6acb03b7a5"}},{"code_sha256_prefix":"b3f21f61e2205323","entry":"RLAgent","repo":"utiasstars/lfgp","repo_kind":"official","path":"rl_sandbox/rl_sandbox/agents/hrl_agents.py","file_url":"https://github.com/utiasstars/lfgp/blob/HEAD/rl_sandbox/rl_sandbox/agents/hrl_agents.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3f21f61e2205323"}},{"code_sha256_prefix":"77e4b3d5c145bd42","entry":"str2tuple","repo":"utiasstars/lfgp","repo_kind":"official","path":"rl_sandbox/rl_sandbox/examples/lfgp/run_lfgp.py","file_url":"https://github.com/utiasstars/lfgp/blob/HEAD/rl_sandbox/rl_sandbox/examples/lfgp/run_lfgp.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77e4b3d5c145bd42"}},{"code_sha256_prefix":"a3689741d2b3ab90","entry":"ACAgent","repo":"utiasstars/lfgp","repo_kind":"official","path":"rl_sandbox/rl_sandbox/agents/hrl_agents.py","file_url":"https://github.com/utiasstars/lfgp/blob/HEAD/rl_sandbox/rl_sandbox/agents/hrl_agents.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3689741d2b3ab90"}},{"code_sha256_prefix":"631da0136bd40912","entry":"SACXAgent","repo":"utiasstars/lfgp","repo_kind":"official","path":"rl_sandbox/rl_sandbox/agents/hrl_agents.py","file_url":"https://github.com/utiasstars/lfgp/blob/HEAD/rl_sandbox/rl_sandbox/agents/hrl_agents.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"631da0136bd40912"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}