{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scaling-laws-for-imitation-learning-in","title":"Scaling Laws for Imitation Learning in Single-Agent Games","arxiv_id":"2307.09423","date":"2023-07-18","proceeding":null,"authors":["Jens Tuyls","Dhruv Madeka","Kari Torkkola","Dean Foster","Karthik Narasimhan","Sham Kakade"],"abstract":"Imitation Learning (IL) is one of the most widely used methods in machine learning. Yet, many works find it is often unable to fully recover the underlying expert behavior, even in constrained environments like single-agent games. However, none of these works deeply investigate the role of scaling up the model and data size. Inspired by recent work in Natural Language Processing (NLP) where \"scaling up\" has resulted in increasingly more capable LLMs, we investigate whether carefully scaling up model and data size can bring similar improvements in the imitation learning setting for single-agent games. We first demonstrate our findings on a variety of Atari games, and thereafter focus on the extremely challenging game of NetHack. In all games, we find that IL loss and mean return scale smoothly with the compute budget (FLOPs) and are strongly correlated, resulting in power laws for training compute-optimal IL agents. Finally, we forecast and train several NetHack agents with IL and find they outperform prior state-of-the-art by 1.5x in all settings. Our work both demonstrates the scaling behavior of imitation learning in a variety of single-agent games, as well as the viability of scaling up current approaches for increasingly capable agents in NetHack, a game that remains elusively hard for current AI systems.","url_abs":"https://arxiv.org/abs/2307.09423v3","url_pdf":"https://arxiv.org/pdf/2307.09423v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scaling-laws-for-imitation-learning-in","repo_url":"https://github.com/princeton-nlp/il-scaling-in-games","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"nethack","task_name":"NetHack"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":null,"method_name":"None"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2307.09423","atlas_url":"https://app.syntology.ai/?focus=2307.09423","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.09423"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/princeton-nlp/il-scaling-in-games","reach":{"status":"ok"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"5a77a04ba19bba16","entry":"eval_model","repo":"princeton-nlp/il-scaling-in-games","repo_kind":"official","path":"il_scale/atari/eval_game.py","file_url":"https://github.com/princeton-nlp/il-scaling-in-games/blob/HEAD/il_scale/atari/eval_game.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a77a04ba19bba16"}},{"code_sha256_prefix":"597f29032b27dac0","entry":"evaluate","repo":"princeton-nlp/il-scaling-in-games","repo_kind":"official","path":"il_scale/atari/eval_return.py","file_url":"https://github.com/princeton-nlp/il-scaling-in-games/blob/HEAD/il_scale/atari/eval_return.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"597f29032b27dac0"}},{"code_sha256_prefix":"aeec3be83c0e22d2","entry":"mask_labels_from_gameids","repo":"princeton-nlp/il-scaling-in-games","repo_kind":"official","path":"il_scale/atari/eval_loss.py","file_url":"https://github.com/princeton-nlp/il-scaling-in-games/blob/HEAD/il_scale/atari/eval_loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aeec3be83c0e22d2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}