{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lad-language-augmented-diffusion-for","title":"Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks","arxiv_id":"2210.15629","date":"2022-10-27","proceeding":null,"authors":["Edwin Zhang","Yujie Lu","Shinda Huang","William Wang","Amy Zhang"],"abstract":"Training generalist agents is difficult across several axes, requiring us to deal with high-dimensional inputs (space), long horizons (time), and generalization to novel tasks. Recent advances with architectures have allowed for improved scaling along one or two of these axes, but are still computationally prohibitive to use. In this paper, we propose to address all three axes by leveraging \\textbf{L}anguage to \\textbf{C}ontrol \\textbf{D}iffusion models as a hierarchical planner conditioned on language (LCD). We effectively and efficiently scale diffusion models for planning in extended temporal, state, and task dimensions to tackle long horizon control problems conditioned on natural language instructions, as a step towards generalist agents. Comparing LCD with other state-of-the-art models on the CALVIN language robotics benchmark finds that LCD outperforms other SOTA methods in multi-task success rates, whilst improving inference speed over other comparable diffusion models by 3.3x~15x. We show that LCD can successfully leverage the unique strength of diffusion models to produce coherent long range plans while addressing their weakness in generating low-level details and control.","url_abs":"https://arxiv.org/abs/2210.15629v4","url_pdf":"https://arxiv.org/pdf/2210.15629v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lad-language-augmented-diffusion-for","repo_url":"https://github.com/ezhang7423/language-control-diffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2210.15629","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.15629"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ezhang7423/language-control-diffusion","reach":null}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eac173f0a5136904","entry":"WeightedL1","repo":"ezhang7423/language-control-diffusion","repo_kind":"official","path":"src/lcd/models/diffusion.py","file_url":"https://github.com/ezhang7423/language-control-diffusion/blob/HEAD/src/lcd/models/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eac173f0a5136904"}},{"code_sha256_prefix":"33d5229f504bb6ae","entry":"WeightedL2","repo":"ezhang7423/language-control-diffusion","repo_kind":"official","path":"src/lcd/models/diffusion.py","file_url":"https://github.com/ezhang7423/language-control-diffusion/blob/HEAD/src/lcd/models/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33d5229f504bb6ae"}},{"code_sha256_prefix":"30d9b14f01180374","entry":"GaussianDiffusion","repo":"ezhang7423/language-control-diffusion","repo_kind":"official","path":"src/lcd/models/diffusion.py","file_url":"https://github.com/ezhang7423/language-control-diffusion/blob/HEAD/src/lcd/models/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30d9b14f01180374"}},{"code_sha256_prefix":"d7c1ac55cf6677ba","entry":"WeightedLoss","repo":"ezhang7423/language-control-diffusion","repo_kind":"official","path":"src/lcd/models/diffusion.py","file_url":"https://github.com/ezhang7423/language-control-diffusion/blob/HEAD/src/lcd/models/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7c1ac55cf6677ba"}},{"code_sha256_prefix":"64ba75a349d16b60","entry":"default_sample_fn","repo":"ezhang7423/language-control-diffusion","repo_kind":"official","path":"src/lcd/models/diffusion.py","file_url":"https://github.com/ezhang7423/language-control-diffusion/blob/HEAD/src/lcd/models/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64ba75a349d16b60"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}