{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/jack-of-all-trades-master-of-some-a-multi","title":"Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent","arxiv_id":"2402.09844","date":"2024-02-15","proceeding":null,"authors":["Quentin Gallouédec","Edward Beeching","Clément Romac","Emmanuel Dellandréa"],"abstract":"The search for a general model that can operate seamlessly across multiple domains remains a key goal in machine learning research. The prevailing methodology in Reinforcement Learning (RL) typically limits models to a single task within a unimodal framework, a limitation that contrasts with the broader vision of a versatile, multi-domain model. In this paper, we present Jack of All Trades (JAT), a transformer-based model with a unique design optimized for handling sequential decision-making tasks and multi-modal data types. The JAT model demonstrates its robust capabilities and versatility by achieving strong performance on very different RL benchmarks, along with promising results on Computer Vision (CV) and Natural Language Processing (NLP) tasks, all using a single set of weights. The JAT model marks a significant step towards more general, cross-domain AI model design, and notably, it is the first model of its kind to be fully open-sourced at https://huggingface.co/jat-project/jat, including a pioneering general-purpose dataset.","url_abs":"https://arxiv.org/abs/2402.09844v3","url_pdf":"https://arxiv.org/pdf/2402.09844v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"jack-of-all-trades-master-of-some-a-multi","repo_url":"https://github.com/huggingface/jat","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2402.09844","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.09844"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/jat","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":5},"by_repo_kind":{"official":{"samples":11,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ce965930fa44e9b4","entry":"compute_ce_loss","repo":"huggingface/jat","repo_kind":"official","path":"jat/modeling_jat.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/modeling_jat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce965930fa44e9b4"}},{"code_sha256_prefix":"9889e7f5f482c648","entry":"compute_mse_loss","repo":"huggingface/jat","repo_kind":"official","path":"jat/modeling_jat.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/modeling_jat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9889e7f5f482c648"}},{"code_sha256_prefix":"50546bcb188a6472","entry":"cyclic_expand_dim","repo":"huggingface/jat","repo_kind":"official","path":"jat/modeling_jat.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/modeling_jat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50546bcb188a6472"}},{"code_sha256_prefix":"a09e847bcb3917cc","entry":"pad","repo":"huggingface/jat","repo_kind":"official","path":"jat/processing_jat.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/processing_jat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a09e847bcb3917cc"}},{"code_sha256_prefix":"cdde81732e084854","entry":"to_tensor","repo":"huggingface/jat","repo_kind":"official","path":"jat/processing_jat.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/processing_jat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cdde81732e084854"}},{"code_sha256_prefix":"92702d141fa25fd0","entry":"truncate","repo":"huggingface/jat","repo_kind":"official","path":"jat/processing_jat.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/processing_jat.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92702d141fa25fd0"}},{"code_sha256_prefix":"9c1882631bb2a9e9","entry":"interleave_datasets","repo":"huggingface/jat","repo_kind":"official","path":"jat/utils_interleave_datasets.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/utils_interleave_datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c1882631bb2a9e9"}},{"code_sha256_prefix":"ff3946a18bfa4ec9","entry":"make_atari","repo":"huggingface/jat","repo_kind":"official","path":"jat/eval/rl/core.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/eval/rl/core.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff3946a18bfa4ec9"}},{"code_sha256_prefix":"6166315cdf0375e4","entry":"make_babyai","repo":"huggingface/jat","repo_kind":"official","path":"jat/eval/rl/core.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/eval/rl/core.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6166315cdf0375e4"}},{"code_sha256_prefix":"a5062b49b9e93ada","entry":"make_metaworld","repo":"huggingface/jat","repo_kind":"official","path":"jat/eval/rl/core.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/eval/rl/core.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a5062b49b9e93ada"}},{"code_sha256_prefix":"4f68b1d9162757de","entry":"normalize","repo":"huggingface/jat","repo_kind":"official","path":"jat/utils.py","file_url":"https://github.com/huggingface/jat/blob/HEAD/jat/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f68b1d9162757de"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}