{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rt-1-robotics-transformer-for-real-world","title":"RT-1: Robotics Transformer for Real-World Control at Scale","arxiv_id":"2212.06817","date":"2022-12-13","proceeding":null,"authors":["Anthony Brohan","Noah Brown","Justice Carbajal","Yevgen Chebotar","Joseph Dabis","Chelsea Finn","Keerthana Gopalakrishnan","Karol Hausman","Alex Herzog","Jasmine Hsu","Julian Ibarz","Brian Ichter","Alex Irpan","Tomas Jackson","Sally Jesmonth","Nikhil J Joshi","Ryan Julian","Dmitry Kalashnikov","Yuheng Kuang","Isabel Leal","Kuang-Huei Lee","Sergey Levine","Yao Lu","Utsav Malla","Deeksha Manjunath","Igor Mordatch","Ofir Nachum","Carolina Parada","Jodilyn Peralta","Emily Perez","Karl Pertsch","Jornell Quiambao","Kanishka Rao","Michael Ryoo","Grecia Salazar","Pannag Sanketi","Kevin Sayed","Jaspiar Singh","Sumedh Sontakke","Austin Stone","Clayton Tan","Huong Tran","Vincent Vanhoucke","Steve Vega","Quan Vuong","Fei Xia","Ted Xiao","Peng Xu","Sichun Xu","Tianhe Yu","Brianna Zitkovich"],"abstract":"By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance. While this capability has been demonstrated in other fields such as computer vision, natural language processing or speech recognition, it remains to be shown in robotics, where the generalization capabilities of the models are particularly critical due to the difficulty of collecting real-world robotic data. We argue that one of the keys to the success of such general robotic models lies with open-ended task-agnostic training, combined with high-capacity architectures that can absorb all of the diverse, robotic data. In this paper, we present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks. The project's website and videos can be found at robotics-transformer1.github.io","url_abs":"https://arxiv.org/abs/2212.06817v2","url_pdf":"https://arxiv.org/pdf/2212.06817v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rt-1-robotics-transformer-for-real-world","repo_url":"https://github.com/google-research/robotics_transformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/robot-manipulation-on-calvin","task":"Robot Manipulation","dataset":"CALVIN","model":"RT-1","rank_in_archive_order":19,"of":19,"metrics":{"avg. sequence length (D to D)":"0.90"},"uses_additional_data":false},{"leaderboard":"/sota/robot-manipulation-on-simpler-env","task":"Robot Manipulation","dataset":"SimplerEnv-Google Robot","model":"RT-1-X","rank_in_archive_order":6,"of":9,"metrics":{"Variant Aggregation":"0.397","Variant Aggregation-Move Near":"0.323","Variant Aggregation-Open/Close Drawer":"0.294","Variant Aggregation-Pick Coke Can":"0.490","Visual Matching":"0.534","Visual Matching-Move Near":"0.317","Visual Matching-Open/Close Drawer":"0.597","Visual Matching-Pick Coke Can":"0.567"},"uses_additional_data":true},{"leaderboard":"/sota/robot-manipulation-on-simplerenv-widow-x","task":"Robot Manipulation","dataset":"SimplerEnv-Widow X","model":"RT-1-X","rank_in_archive_order":6,"of":7,"metrics":{"Average":"0.011","Put Carrot on Plate":"0.042","Put Spoon on Towel":"0.000","Stack Green Block on Yellow Block":"0.000"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2212.06817","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.06817"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/robotics_transformer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8c721abb2a535d2c","entry":"correct_pad","repo":"google-research/robotics_transformer","repo_kind":"official","path":"film_efficientnet/film_efficientnet_encoder.py","file_url":"https://github.com/google-research/robotics_transformer/blob/HEAD/film_efficientnet/film_efficientnet_encoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c721abb2a535d2c"}},{"code_sha256_prefix":"b4f4f969a90d69b8","entry":"gelu","repo":"google-research/robotics_transformer","repo_kind":"official","path":"tokenizers/token_learner.py","file_url":"https://github.com/google-research/robotics_transformer/blob/HEAD/tokenizers/token_learner.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b4f4f969a90d69b8"}},{"code_sha256_prefix":"00fc88bba1e1f957","entry":"observations_list","repo":"google-research/robotics_transformer","repo_kind":"official","path":"transformer_network_test_set_up.py","file_url":"https://github.com/google-research/robotics_transformer/blob/HEAD/transformer_network_test_set_up.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00fc88bba1e1f957"}},{"code_sha256_prefix":"7e6302fbfec92d53","entry":"obtain_input_shape","repo":"google-research/robotics_transformer","repo_kind":"official","path":"film_efficientnet/film_efficientnet_encoder.py","file_url":"https://github.com/google-research/robotics_transformer/blob/HEAD/film_efficientnet/film_efficientnet_encoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7e6302fbfec92d53"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}