{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/plan-your-target-and-learn-your-skills","title":"Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization","arxiv_id":"2203.02214","date":"2022-03-04","proceeding":null,"authors":["Minghuan Liu","Zhengbang Zhu","Yuzheng Zhuang","Weinan Zhang","Jianye Hao","Yong Yu","Jun Wang"],"abstract":"Recent progress in state-only imitation learning extends the scope of applicability of imitation learning to real-world settings by relieving the need for observing expert actions. However, existing solutions only learn to extract a state-to-action mapping policy from the data, without considering how the expert plans to the target. This hinders the ability to leverage demonstrations and limits the flexibility of the policy. In this paper, we introduce Decoupled Policy Optimization (DePO), which explicitly decouples the policy as a high-level state planner and an inverse dynamics model. With embedded decoupled policy gradient and generative adversarial training, DePO enables knowledge transfer to different action spaces or state transition dynamics, and can generalize the planner to out-of-demonstration state regions. Our in-depth experimental analysis shows the effectiveness of DePO on learning a generalized target state planner while achieving the best imitation performance. We demonstrate the appealing usage of DePO for transferring across different tasks by pre-training, and the potential for co-training agents with various skills.","url_abs":"https://arxiv.org/abs/2203.02214v5","url_pdf":"https://arxiv.org/pdf/2203.02214v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"plan-your-target-and-learn-your-skills","repo_url":"https://github.com/apexrl/DePO","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"plan-your-target-and-learn-your-skills","repo_url":"https://github.com/apexrl/depo_ngsim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2203.02214","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.02214"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/apexrl/DePO","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/apexrl/DePO_NGSIM","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/apexrl/depo_ngsim","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"0f86930940568c69","entry":"PathBuilder","repo":"apexrl/depo_ngsim","repo_kind":"official","path":"ILSwiss/rlkit/samplers/dpo_sampler.py","file_url":"https://github.com/apexrl/depo_ngsim/blob/HEAD/ILSwiss/rlkit/samplers/dpo_sampler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f86930940568c69"}},{"code_sha256_prefix":"d0cb5035d9ad8713","entry":"stack_list","repo":"apexrl/depo_ngsim","repo_kind":"official","path":"ILSwiss/rlkit/samplers/dpo_sampler.py","file_url":"https://github.com/apexrl/depo_ngsim/blob/HEAD/ILSwiss/rlkit/samplers/dpo_sampler.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d0cb5035d9ad8713"}},{"code_sha256_prefix":"7afbe1be707c1e51","entry":"AgentPathBuilder","repo":"apexrl/depo_ngsim","repo_kind":"official","path":"ILSwiss/rlkit/samplers/dpo_sampler.py","file_url":"https://github.com/apexrl/depo_ngsim/blob/HEAD/ILSwiss/rlkit/samplers/dpo_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7afbe1be707c1e51"}},{"code_sha256_prefix":"75722dcc85782a74","entry":"DPOPathSampler","repo":"apexrl/depo_ngsim","repo_kind":"official","path":"ILSwiss/rlkit/samplers/dpo_sampler.py","file_url":"https://github.com/apexrl/depo_ngsim/blob/HEAD/ILSwiss/rlkit/samplers/dpo_sampler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75722dcc85782a74"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}