{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/skilldiffuser-interpretable-hierarchical","title":"SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution","arxiv_id":"2312.11598","date":"2023-12-18","proceeding":"CVPR 2024 1","authors":["Zhixuan Liang","Yao Mu","Hengbo Ma","Masayoshi Tomizuka","Mingyu Ding","Ping Luo"],"abstract":"Diffusion models have demonstrated strong potential for robotic trajectory planning. However, generating coherent trajectories from high-level instructions remains challenging, especially for long-range composition tasks requiring multiple sequential skills. We propose SkillDiffuser, an end-to-end hierarchical planning framework integrating interpretable skill learning with conditional diffusion planning to address this problem. At the higher level, the skill abstraction module learns discrete, human-understandable skill representations from visual observations and language instructions. These learned skill embeddings are then used to condition the diffusion model to generate customized latent trajectories aligned with the skills. This allows generating diverse state trajectories that adhere to the learnable skills. By integrating skill learning with conditional trajectory generation, SkillDiffuser produces coherent behavior following abstract instructions across diverse tasks. Experiments on multi-task robotic manipulation benchmarks like Meta-World and LOReL demonstrate state-of-the-art performance and human-interpretable skill representations from SkillDiffuser. More visualization results and information could be found on our website.","url_abs":"https://arxiv.org/abs/2312.11598v3","url_pdf":"https://arxiv.org/pdf/2312.11598v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"skilldiffuser-interpretable-hierarchical","repo_url":"https://github.com/Liang-ZX/SkillDiffuser","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok"}}],"tasks":[{"task_slug":"trajectory-planning","task_name":"Trajectory Planning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[{"slug":"short-metaworld","name":"short-MetaWorld","full_name":"MetaWorld with 20 step trajectories"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2312.11598","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.11598"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Liang-ZX/SkillDiffuser","reach":{"status":"ok"}}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"15eab9eebd751706","entry":"read_file","repo":"Liang-ZX/SkillDiffuser","repo_kind":"official","path":"skilldiffuser/hrl/expert_dataset.py","file_url":"https://github.com/Liang-ZX/SkillDiffuser/blob/HEAD/skilldiffuser/hrl/expert_dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15eab9eebd751706"}},{"code_sha256_prefix":"6bad17d71a8fc8bb","entry":"get_action","repo":"Liang-ZX/SkillDiffuser","repo_kind":"official","path":"skilldiffuser/hrl/eval_orig.py","file_url":"https://github.com/Liang-ZX/SkillDiffuser/blob/HEAD/skilldiffuser/hrl/eval_orig.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6bad17d71a8fc8bb"}},{"code_sha256_prefix":"1b9f027b83323ba0","entry":"load_babyai_data","repo":"Liang-ZX/SkillDiffuser","repo_kind":"official","path":"skilldiffuser/hrl/expert_dataset.py","file_url":"https://github.com/Liang-ZX/SkillDiffuser/blob/HEAD/skilldiffuser/hrl/expert_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1b9f027b83323ba0"}},{"code_sha256_prefix":"f2bf1c69ffe9deea","entry":"load_trajectories","repo":"Liang-ZX/SkillDiffuser","repo_kind":"official","path":"skilldiffuser/hrl/expert_dataset.py","file_url":"https://github.com/Liang-ZX/SkillDiffuser/blob/HEAD/skilldiffuser/hrl/expert_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f2bf1c69ffe9deea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}