Methods › Reinforcement Learning › Video Game Models › AlphaStar
DeepMind AlphaStar
AlphaStar
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
AlphaStar is a reinforcement learning agent for tackling the game of Starcraft II. It learns a policy π_θ(aₜ|sₜ, z) = P[aₜ|sₜ, z] using a neural network for parameters θ that receives observations sₜ = (o_(1:t), a_(1:t-1)) as inputs and chooses actions as outputs. Additionally, the policy conditions on a statistic z that summarizes a strategy sampled from human data such as a build order [1].
AlphaStar uses numerous types of architecture to incorporate different types of features. Observations of player and enemy units are processed with a Transformer. Scatter connections are used to integrate spatial and non-spatial information. The temporal sequence of observations is processed by a core LSTM. Minimap features are extracted with a Residual Network. To manage the combinatorial action space, the agent uses an autoregressive policy and a recurrent pointer network.
The agent is trained first with supervised learning from human replays. Parameters are subsequently trained using reinforcement learning that maximizes the win rate against opponents. The RL algorithm is based on a policy-gradient algorithm similar to actor-critic. Updates are performed asynchronously and off-policy. To deal with this, a combination of TD(λ) and V-trace are used, as well as a new self-imitation algorithm (UPGO).
Lastly, to address game-theoretic challenges, AlphaStar is trained with league training to try to approximate a fictitious self-play (FSP) setting which avoids cycles by computing a best response against a uniform mixture of all previous policies. The league of potential opponents includes a diverse range of agents, including policies from current and previous agents.
Image Credit: Yekun Chai
References
- Chai, Yekun. "Deciphering AlphaStar on StarCraft II." (2019). https://cyk1337.github.io/notes/2019/07/21/RL/DRL/Decipher-AlphaStar-on-StarCraft-II/
Code Implementation
- https://github.com/opendilab/DI-star
Papers archive 2025-07-28
10 shown of 10, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach 19 Dec 2023 · 1 repository · arXiv:2312.11865Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
-
A Robust and Opponent-Aware League Training Method for StarCraft II 21 Sep 2023 · 0 repositories
-
AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning 7 Aug 2023 · 1 repository · arXiv:2308.03526Syntology ran 4 of 4 samples · 0 unverified
-
On Efficient Reinforcement Learning for Full-length Game of StarCraft II 23 Sep 2022 · 2 repositories · arXiv:2209.11553Syntology ran 0 of 10 samples · 10 unverified
-
AI in Human-computer Gaming: Techniques, Challenges and Opportunities 15 Nov 2021 · 0 repositories · arXiv:2111.07631
-
Rethinking of AlphaStar 7 Aug 2021 · 2 repositories · arXiv:2108.03452
-
An Introduction of mini-AlphaStar 14 Apr 2021 · 1 repository · arXiv:2104.06890
-
Deep Reinforcement Learning with Function Properties in Mean Reversion Strategies 9 Jan 2021 · 1 repository · arXiv:2101.03418
-
TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game 27 Nov 2020 · 1 repository · arXiv:2011.13729
-
AlphaStar: An Evolutionary Computation Perspective 5 Feb 2019 · 0 repositories · arXiv:1902.01724
Tasks archive 2025-07-28
15 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections