Methods › Reinforcement Learning › Distributed Reinforcement Learning › Ape-X
Ape-X
Introduced by Dan Horgan et al. in Distributed Prioritized Experience Replay
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Ape-X is a distributed architecture for deep reinforcement learning. The algorithm decouples acting from learning: the actors interact with their own instances of the environment by selecting actions according to a shared neural network, and accumulate the resulting experience in a shared experience replay memory; the learner replays samples of experience and updates the neural network. The architecture relies on prioritized experience replay to focus only on the most significant data generated by the actors.
In contrast to Gorila, Ape-X uses a shared, centralized replay memory, and instead of sampling uniformly, it prioritizes, to sample the most useful data more often. All communications are batched with the centralized replay, increasing the efficiency and throughput at the cost of some latency. And by learning off-policy, Ape-X has the ability to combine data from many distributed actors, by giving the different actors different exploration policies, broadening the diversity of the experience they jointly encounter.
Papers archive 2025-07-28
10 shown of 10, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Safety Margins for Reinforcement Learning 25 Jul 2023 · 0 repositories · arXiv:2307.13642
-
Parallel Reinforcement Learning Simulation for Visual Quadrotor Navigation 22 Sep 2022 · 0 repositories · arXiv:2209.11094
-
A coevolutionary approach to deep multi-agent reinforcement learning 12 Apr 2021 · 1 repository · arXiv:2104.05610
-
A Learning Approach to Robot-Agnostic Force-Guided High Precision Assembly 15 Oct 2020 · 0 repositories · arXiv:2010.08052
-
Dynamic Experience Replay 4 Mar 2020 · 0 repositories · arXiv:2003.02372
-
Google Research Football: A Novel Reinforcement Learning Environment 25 Jul 2019 · 1 repository · arXiv:1907.11180Syntology ran 0 of 4 samples · 4 unverified
-
Macro action selection with deep reinforcement learning in StarCraft 2 Dec 2018 · 1 repository · arXiv:1812.00336
-
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution 9 Jul 2018 · 24 repositories · arXiv:1807.03247Syntology ran 4 of 5 samples · 1 unverified
-
Deep Curiosity Search: Intra-Life Exploration Can Improve Performance on Challenging Deep Reinforcement Learning Problems 1 Jun 2018 · 0 repositories · arXiv:1806.00553
-
Distributed Prioritized Experience Replay 2 Mar 2018 · 15 repositories · arXiv:1803.00933Syntology ran 0 of 15 samples · 15 unverified
Tasks archive 2025-07-28
16 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections