{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/starcraft-ii-a-new-challenge-for","title":"StarCraft II: A New Challenge for Reinforcement Learning","arxiv_id":"1708.04782","date":"2017-08-16","proceeding":null,"authors":["Oriol Vinyals","Timo Ewalds","Sergey Bartunov","Petko Georgiev","Alexander Sasha Vezhnevets","Michelle Yeo","Alireza Makhzani","Heinrich Küttler","John Agapiou","Julian Schrittwieser","John Quan","Stephen Gaffney","Stig Petersen","Karen Simonyan","Tom Schaul","Hado van Hasselt","David Silver","Timothy Lillicrap","Kevin Calderone","Paul Keet","Anthony Brunasso","David Lawrence","Anders Ekermo","Jacob Repp","Rodney Tsing"],"abstract":"This paper introduces SC2LE (StarCraft II Learning Environment), a\nreinforcement learning environment based on the StarCraft II game. This domain\nposes a new grand challenge for reinforcement learning, representing a more\ndifficult class of problems than considered in most prior work. It is a\nmulti-agent problem with multiple players interacting; there is imperfect\ninformation due to a partially observed map; it has a large action space\ninvolving the selection and control of hundreds of units; it has a large state\nspace that must be observed solely from raw input feature planes; and it has\ndelayed credit assignment requiring long-term strategies over thousands of\nsteps. We describe the observation, action, and reward specification for the\nStarCraft II domain and provide an open source Python-based interface for\ncommunicating with the game engine. In addition to the main game maps, we\nprovide a suite of mini-games focusing on different elements of StarCraft II\ngameplay. For the main game maps, we also provide an accompanying dataset of\ngame replay data from human expert players. We give initial baseline results\nfor neural networks trained from this data to predict game outcomes and player\nactions. Finally, we present initial baseline results for canonical deep\nreinforcement learning agents applied to the StarCraft II domain. On the\nmini-games, these agents learn to achieve a level of play that is comparable to\na novice player. However, when trained on the main game, these agents are\nunable to make significant progress. Thus, SC2LE offers a new and challenging\nenvironment for exploring deep reinforcement learning algorithms and\narchitectures.","url_abs":"http://arxiv.org/abs/1708.04782v1","url_pdf":"http://arxiv.org/pdf/1708.04782v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/deepmind/pysc2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/4rChon/NL-FuN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/Teslatic/SC2-Freiburg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/ericborn/binarybot","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/google-deepmind/pysc2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/inoryy/reaver","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/nicoladainese96/SC2-RL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/raccoon831012/StartCraft2-RL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/tuomaso/SC2LE-implementation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"starcraft-ii-a-new-challenge-for","repo_url":"https://github.com/NervanaSystems/coach","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"real-time-strategy-games","task_name":"Real-Time Strategy Games"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"starcraft","task_name":"Starcraft"},{"task_slug":"starcraft-ii","task_name":"Starcraft II"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[{"slug":"starcraft-ii-learning-environment","name":"StarCraft II Learning Environment","full_name":"StarCraft II Learning Environment"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/starcraft-ii-on-collectmineralshards","task":"Starcraft II","dataset":"CollectMineralShards","model":"FullyConv LSTM","rank_in_archive_order":1,"of":1,"metrics":{"Max Score":"137"},"uses_additional_data":false},{"leaderboard":"/sota/starcraft-ii-on-movetobeacon","task":"Starcraft II","dataset":"MoveToBeacon","model":"FullyConv LSTM","rank_in_archive_order":1,"of":1,"metrics":{"Max Score":"35"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1708.04782","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1708.04782"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/pysc2","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nicoladainese96/SC2-RL","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Teslatic/SC2-Freiburg","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ericborn/binarybot","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/pysc2","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tuomaso/SC2LE-implementation","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/raccoon831012/StartCraft2-RL","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/4rChon/NL-FuN","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/inoryy/reaver","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NervanaSystems/coach","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":3,"ran_violates":2},"by_repo_kind":{"listed":{"samples":4,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"e9123707cba84b2c","entry":"compute_baseline_loss","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e9123707cba84b2c"}},{"code_sha256_prefix":"fa62aa3ffdea8919","entry":"compute_policy_gradient_loss","repo":"nicoladainese96/SC2-RL","repo_kind":"listed","path":"monobeast.py","file_url":"https://github.com/nicoladainese96/SC2-RL/blob/HEAD/monobeast.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa62aa3ffdea8919"}},{"code_sha256_prefix":"7d251f4894d76c43","entry":"crop_and_deduplicate_names","repo":"google-deepmind/pysc2","repo_kind":"listed","path":"pysc2/env/sc2_env.py","file_url":"https://github.com/google-deepmind/pysc2/blob/HEAD/pysc2/env/sc2_env.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7d251f4894d76c43"}},{"code_sha256_prefix":"42e509fdebb7a09d","entry":"get_default","repo":"google-deepmind/pysc2","repo_kind":"listed","path":"pysc2/env/sc2_env.py","file_url":"https://github.com/google-deepmind/pysc2/blob/HEAD/pysc2/env/sc2_env.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42e509fdebb7a09d"}},{"code_sha256_prefix":"5d26ef3681029c3e","entry":"to_list","repo":"google-deepmind/pysc2","repo_kind":"listed","path":"pysc2/env/sc2_env.py","file_url":"https://github.com/google-deepmind/pysc2/blob/HEAD/pysc2/env/sc2_env.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d26ef3681029c3e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}