{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-generalist-agent","title":"A Generalist Agent","arxiv_id":"2205.06175","date":"2022-05-12","proceeding":"DeepMind 2022 5","authors":["Scott Reed","Konrad Zolna","Emilio Parisotto","Sergio Gomez Colmenarejo","Alexander Novikov","Gabriel Barth-Maron","Mai Gimenez","Yury Sulsky","Jackie Kay","Jost Tobias Springenberg","Tom Eccles","Jake Bruce","Ali Razavi","Ashley Edwards","Nicolas Heess","Yutian Chen","Raia Hadsell","Oriol Vinyals","Mahyar Bordbar","Nando de Freitas"],"abstract":"Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.","url_abs":"https://arxiv.org/abs/2205.06175v3","url_pdf":"https://arxiv.org/pdf/2205.06175v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-generalist-agent","repo_url":"https://github.com/LAS1520/Gato-A-Generalist-Agent","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-generalist-agent","repo_url":"https://github.com/ManifoldRG/gato-control","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"a-generalist-agent","repo_url":"https://github.com/OrigamiDream/gato","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"skill-generalization","task_name":"Skill Generalization"},{"task_slug":"skill-mastery","task_name":"Skill Mastery"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/skill-generalization-on-rgb-stacking","task":"Skill Generalization","dataset":"RGB-Stacking","model":"Gato","rank_in_archive_order":1,"of":2,"metrics":{"Average":"50.2","Group 1":"24.5","Group 2":"33","Group 3":"50.5","Group 4":"76.5","Group 5":"66.5"},"uses_additional_data":false},{"leaderboard":"/sota/skill-mastery-on-rgb-stacking","task":"Skill Mastery","dataset":"RGB-Stacking","model":"Gato","rank_in_archive_order":1,"of":2,"metrics":{"Average":"75.6","Group 1":"58","Group 2":"57.6","Group 3":"78.5","Group 4":"89","Group 5":"95.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.06175","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.06175"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OrigamiDream/gato","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ManifoldRG/gato-control","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LAS1520/Gato-A-Generalist-Agent","reach":null}],"summary":{"unverified":2},"by_repo_kind":{"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0615ee0ff9470f2f","entry":"mu_law_encode","repo":"OrigamiDream/gato","repo_kind":"listed","path":"gato/models/tokenizers.py","file_url":"https://github.com/OrigamiDream/gato/blob/HEAD/gato/models/tokenizers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0615ee0ff9470f2f"}},{"code_sha256_prefix":"58f69cafe9ba15e5","entry":"tokenize_continuous_values","repo":"OrigamiDream/gato","repo_kind":"listed","path":"gato/models/tokenizers.py","file_url":"https://github.com/OrigamiDream/gato/blob/HEAD/gato/models/tokenizers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58f69cafe9ba15e5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}