{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dita-scaling-diffusion-transformer-for","title":"Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy","arxiv_id":"2503.19757","date":"2025-03-25","proceeding":null,"authors":["Zhi Hou","Tianyi Zhang","Yuwen Xiong","Haonan Duan","Hengjun Pu","Ronglei Tong","Chengyang Zhao","Xizhou Zhu","Yu Qiao","Jifeng Dai","Yuntao Chen"],"abstract":"While recent vision-language-action models trained on diverse robot datasets exhibit promising generalization capabilities with limited in-domain data, their reliance on compact action heads to predict discretized or continuous actions constrains adaptability to heterogeneous action spaces. We present Dita, a scalable framework that leverages Transformer architectures to directly denoise continuous action sequences through a unified multimodal diffusion process. Departing from prior methods that condition denoising on fused embeddings via shallow networks, Dita employs in-context conditioning -- enabling fine-grained alignment between denoised actions and raw visual tokens from historical observations. This design explicitly models action deltas and environmental nuances. By scaling the diffusion action denoiser alongside the Transformer's scalability, Dita effectively integrates cross-embodiment datasets across diverse camera perspectives, observation scenes, tasks, and action spaces. Such synergy enhances robustness against various variances and facilitates the successful execution of long-horizon tasks. Evaluations across extensive benchmarks demonstrate state-of-the-art or comparative performance in simulation. Notably, Dita achieves robust real-world adaptation to environmental variances and complex long-horizon tasks through 10-shot finetuning, using only third-person camera inputs. The architecture establishes a versatile, lightweight and open-source baseline for generalist robot policy learning. Project Page: https://robodita.github.io.","url_abs":"https://arxiv.org/abs/2503.19757v1","url_pdf":"https://arxiv.org/pdf/2503.19757v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dita-scaling-diffusion-transformer-for","repo_url":"https://github.com/RoboDita/Dita","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"},{"task_slug":"vision-language-action","task_name":"Vision-Language-Action"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/robot-manipulation-on-simpler-env","task":"Robot Manipulation","dataset":"SimplerEnv-Google Robot","model":"Dita-300M","rank_in_archive_order":3,"of":9,"metrics":{"Variant Aggregation":"0.652","Variant Aggregation-Move Near":"0.730","Variant Aggregation-Open/Close Drawer":"0.370","Variant Aggregation-Pick Coke Can":"0.855","Visual Matching":"0.687","Visual Matching-Move Near":"0.760","Visual Matching-Open/Close Drawer":"0.463","Visual Matching-Pick Coke Can":"0.837"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.19757","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.19757"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/RoboDita/Dita","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3cab76f0744f529e","entry":"adjust_learning_rate","repo":"RoboDita/Dita","repo_kind":"listed","path":"scripts/train_diffusion_oxe.py","file_url":"https://github.com/RoboDita/Dita/blob/HEAD/scripts/train_diffusion_oxe.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3cab76f0744f529e"}},{"code_sha256_prefix":"b792cc83a31ea258","entry":"dict_to_gpu","repo":"RoboDita/Dita","repo_kind":"listed","path":"scripts/finetune_realdata.py","file_url":"https://github.com/RoboDita/Dita/blob/HEAD/scripts/finetune_realdata.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b792cc83a31ea258"}},{"code_sha256_prefix":"a2f935d170b983fd","entry":"reduce_and_average","repo":"RoboDita/Dita","repo_kind":"listed","path":"scripts/train_diffusion_oxe.py","file_url":"https://github.com/RoboDita/Dita/blob/HEAD/scripts/train_diffusion_oxe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2f935d170b983fd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}