Papers › Macro action selection with deep reinforcement learning in StarCraft

Macro action selection with deep reinforcement learning in StarCraft

2 Dec 2018arXiv:1812.00336archive 2025-07-28

Sijia Xu, Hongyu Kuang, Zhi Zhuang, Renjie Hu, Yang Liu, Huyang Sun

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

Bilibili/LastOrder officialmentioned in papermentioned on GitHubtf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Deep Reinforcement Learning Real-Time Strategy Games Reinforcement Learning Reinforcement Learning (RL)Starcraft reinforcement-learning

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Ape-X Ape-X DQN Convolution DQN Dense Connections Double Q-learning Dueling Network N-step Returns Prioritized Experience Replay Q-Learning

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections