{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-diffusion-transformer-policies-with","title":"Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learning","arxiv_id":"2412.12953","date":"2024-12-17","proceeding":null,"authors":["Moritz Reuss","Jyothish Pari","Pulkit Agrawal","Rudolf Lioutikov"],"abstract":"Diffusion Policies have become widely used in Imitation Learning, offering several appealing properties, such as generating multimodal and discontinuous behavior. As models are becoming larger to capture more complex capabilities, their computational demands increase, as shown by recent scaling laws. Therefore, continuing with the current architectures will present a computational roadblock. To address this gap, we propose Mixture-of-Denoising Experts (MoDE) as a novel policy for Imitation Learning. MoDE surpasses current state-of-the-art Transformer-based Diffusion Policies while enabling parameter-efficient scaling through sparse experts and noise-conditioned routing, reducing both active parameters by 40% and inference costs by 90% via expert caching. Our architecture combines this efficient scaling with noise-conditioned self-attention mechanism, enabling more effective denoising across different noise levels. MoDE achieves state-of-the-art performance on 134 tasks in four established imitation learning benchmarks (CALVIN and LIBERO). Notably, by pretraining MoDE on diverse robotics data, we achieve 4.01 on CALVIN ABC and 0.95 on LIBERO-90. It surpasses both CNN-based and Transformer Diffusion Policies by an average of 57% across 4 benchmarks, while using 90% fewer FLOPs and fewer active parameters compared to default Diffusion Transformer architectures. Furthermore, we conduct comprehensive ablations on MoDE's components, providing insights for designing efficient and scalable Transformer architectures for Diffusion Policies. Code and demonstrations are available at https://mbreuss.github.io/MoDE_Diffusion_Policy/.","url_abs":"https://arxiv.org/abs/2412.12953v1","url_pdf":"https://arxiv.org/pdf/2412.12953v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-diffusion-transformer-policies-with","repo_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[{"method_slug":"abc","method_name":"ABC"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/on-libero-10","task":"","dataset":"LIBERO-10","model":"MoDE","rank_in_archive_order":1,"of":1,"metrics":{"Average Success Rate":"0.94"},"uses_additional_data":true},{"leaderboard":"/sota/on-libero-90","task":"","dataset":"LIBERO-90","model":"MoDE","rank_in_archive_order":1,"of":1,"metrics":{"Average Success Rate":"0.95"},"uses_additional_data":true},{"leaderboard":"/sota/robot-manipulation-on-calvin","task":"Robot Manipulation","dataset":"CALVIN","model":"MoDE","rank_in_archive_order":7,"of":19,"metrics":{"avg. sequence length (D to D)":"4.01"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-generalization-on-calvin","task":"Zero-shot Generalization","dataset":"CALVIN","model":"MoDE","rank_in_archive_order":2,"of":5,"metrics":{"Avg. sequence length":"4.01"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.12953","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.12953"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":2,"ran":2,"ran_draft_wrong":1,"unverified":6},"by_repo_kind":{"listed":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"30befb7e4327e615","entry":"append_dims","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/models/edm_diffusion/utils.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/models/edm_diffusion/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":2,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30befb7e4327e615"}},{"code_sha256_prefix":"43f2167b019a5fb7","entry":"append_zero","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/models/edm_diffusion/gc_sampling.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/models/edm_diffusion/gc_sampling.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43f2167b019a5fb7"}},{"code_sha256_prefix":"205e97641ece09f8","entry":"get_sigmas_exponential","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/models/edm_diffusion/gc_sampling.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/models/edm_diffusion/gc_sampling.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"205e97641ece09f8"}},{"code_sha256_prefix":"a3ae2e8a897b4d99","entry":"get_sigmas_karras","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/models/edm_diffusion/gc_sampling.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/models/edm_diffusion/gc_sampling.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3ae2e8a897b4d99"}},{"code_sha256_prefix":"ec8364ee198f15b3","entry":"rand_log_normal","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/models/edm_diffusion/utils.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/models/edm_diffusion/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec8364ee198f15b3"}},{"code_sha256_prefix":"68787b0149b5e4bf","entry":"load_npz","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/datasets/disk_dataset.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/datasets/disk_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68787b0149b5e4bf"}},{"code_sha256_prefix":"45af63efa9933397","entry":"load_pkl","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/datasets/disk_dataset.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/datasets/disk_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45af63efa9933397"}},{"code_sha256_prefix":"6f1fce523ddeea43","entry":"return_time_sigma_embedding_model","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/models/edm_diffusion/utils.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/models/edm_diffusion/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f1fce523ddeea43"}},{"code_sha256_prefix":"0e68bc534d7f6740","entry":"setup_callbacks","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/training_calvin.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/training_calvin.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e68bc534d7f6740"}},{"code_sha256_prefix":"765bd1c53cf958eb","entry":"setup_logger","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/training_calvin.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/training_calvin.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"765bd1c53cf958eb"}},{"code_sha256_prefix":"38618aea799ddc0e","entry":"split_trajectories","repo":"intuitive-robots/MoDE_Diffusion_Policy","repo_kind":"listed","path":"mode/datasets/libero_data_module.py","file_url":"https://github.com/intuitive-robots/MoDE_Diffusion_Policy/blob/HEAD/mode/datasets/libero_data_module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38618aea799ddc0e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}