Methods › Reinforcement Learning › Policy Gradient Methods › A3C
A3C
Introduced by Volodymyr Mnih et al. in Asynchronous Methods for Deep Reinforcement Learning
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
A3C, Asynchronous Advantage Actor Critic, is a policy gradient algorithm in reinforcement learning that maintains a policy π(aₜ|sₜ; θ) and an estimate of the value function V(sₜ; θᵥ). It operates in the forward view and uses a mix of n-step returns to update both the policy and the value-function. The policy and the value function are updated after every tₘₐₓ actions or when a terminal state is reached. The update performed by the algorithm can be seen as ∇_(θ′)logπ(aₜ|sₜ; θ′)A(sₜ, aₜ; θ, θᵥ) where A(sₜ, aₜ; θ, θᵥ) is an estimate of the advantage function given by:
∑ᵏ⁻¹ᵢ₌₀γⁱrₜ₊ᵢ + γᵏV(sₜ₊ₖ; θᵥ) - V(sₜ; θᵥ)
where k can vary from state to state and is upper-bounded by tₘₐₓ.
The critics in A3C learn the value function while multiple actors are trained in parallel and get synced with global parameters every so often. The gradients are accumulated as part of training for stability - this is like parallelized stochastic gradient descent.
Note that while the parameters θ of the policy and θᵥ of the value function are shown as being separate for generality, we always share some of the parameters in practice. We typically use a convolutional neural network that has one softmax output for the policy π(aₜ|sₜ; θ) and one linear output for the value function V(sₜ; θᵥ), with all non-output layers shared.
Papers archive 2025-07-28
30 shown of 57, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Detecting and Mitigating Reward Hacking in Reinforcement Learning Systems: A Comprehensive Empirical Study 8 Jul 2025 · 0 repositories · arXiv:2507.05619
-
Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach 7 May 2025 · 0 repositories · arXiv:2505.04148
-
Intelligent Task Scheduling for Microservices via A3C-Based Reinforcement Learning 1 May 2025 · 0 repositories · arXiv:2505.00299
-
Demand-Aware Beam Hopping and Power Allocation for Load Balancing in Digital Twin empowered LEO Satellite Networks 29 Oct 2024 · 0 repositories · arXiv:2411.08896
-
Survival of the Fittest: Evolutionary Adaptation of Policies for Environmental Shifts 22 Oct 2024 · 0 repositories · arXiv:2410.19852
-
Physical Informed-Inspired Deep Reinforcement Learning Based Bi-Level Programming for Microgrid Scheduling 15 Oct 2024 · 0 repositories · arXiv:2410.11932
-
Criticality and Safety Margins for Reinforcement Learning 26 Sep 2024 · 0 repositories · arXiv:2409.18289
-
Evaluation of Reinforcement Learning for Autonomous Penetration Testing using A3C, Q-learning and DQN 22 Jul 2024 · 0 repositories · arXiv:2407.15656
-
A Deep Reinforcement Learning Approach for Trading Optimization in the Forex Market with Multi-Agent Asynchronous Distribution 30 May 2024 · 0 repositories · arXiv:2405.19982
-
Sum Throughput Maximization in Multi-BD Symbiotic Radio NOMA Network Assisted by Active-STAR-RIS 16 Jan 2024 · 0 repositories · arXiv:2401.08301
-
Learning Actions and Control of Focus of Attention with a Log-Polar-like Sensor 22 Sep 2023 · 0 repositories · arXiv:2309.12634
-
Safety Margins for Reinforcement Learning 25 Jul 2023 · 0 repositories · arXiv:2307.13642
-
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages 2 Jun 2023 · 1 repository · arXiv:2306.01460
-
Double A3C: Deep Reinforcement Learning on OpenAI Gym Games 4 Mar 2023 · 0 repositories · arXiv:2303.02271
-
Reinforcement Learning for Molecular Dynamics Optimization: A Stochastic Pontryagin Maximum Principle Approach 6 Dec 2022 · 1 repository · arXiv:2212.03320
-
Point Cloud Scene Completion with Joint Color and Semantic Estimation from Single RGB-D Image 12 Oct 2022 · 0 repositories · arXiv:2210.05891
-
Comparing Deep Reinforcement Learning Algorithms in Two-Echelon Supply Chains 20 Apr 2022 · 1 repository · arXiv:2204.09603
-
RL-CoSeg : A Novel Image Co-Segmentation Algorithm with Deep Reinforcement Learning 12 Apr 2022 · 0 repositories · arXiv:2204.05951
-
Learning Reward Machines: A Study in Partially Observable Reinforcement Learning 17 Dec 2021 · 0 repositories · arXiv:2112.09477
-
Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning 6 Mar 2021 · 0 repositories · arXiv:2103.04067
-
A review of motion planning algorithms for intelligent robotics 4 Feb 2021 · 0 repositories · arXiv:2102.02376
-
Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup 31 Dec 2020 · 0 repositories · arXiv:2012.15511
-
Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networks 1 Sep 2020 · 1 repository · arXiv:2009.02186
-
Lagrangian Duality in Reinforcement Learning 20 Jul 2020 · 0 repositories · arXiv:2007.09998
-
PFPN: Continuous Control of Physically Simulated Characters using Particle Filtering Policy Network 16 Mar 2020 · 1 repository · arXiv:2003.06959
-
Explore and Exploit with Heterotic Line Bundle Models 10 Mar 2020 · 1 repository · arXiv:2003.04817
-
Fully Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networks 1 Mar 2020 · 0 repositories · arXiv:2003.00433
-
A Visual Communication Map for Multi-Agent Deep Reinforcement Learning 27 Feb 2020 · 0 repositories · arXiv:2002.11882
-
Intelligent Roundabout Insertion using Deep Reinforcement Learning 3 Jan 2020 · 0 repositories · arXiv:2001.00786
-
Intelligent Coordination among Multiple Traffic Intersections Using Multi-Agent Reinforcement Learning 9 Dec 2019 · 0 repositories · arXiv:1912.03851
Tasks archive 2025-07-28
20 shown of 61 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections