Methods › General › Regularization › Target Policy Smoothing
Target Policy Smoothing
Introduced by Scott Fujimoto et al. in Addressing Function Approximation Error in Actor-Critic Methods
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Target Policy Smoothing is a regularization strategy for the value function in reinforcement learning. Deterministic policies can overfit to narrow peaks in the value estimate, making them highly susceptible to functional approximation error, increasing the variance of the target. To reduce this variance, target policy smoothing adds a small amount of random noise to the target policy and averages over mini-batches - approximating a SARSA-like expectation/integral.
The modified target update is:
y = r + γQ_(θ′)(s′, π_(θ′)(s′) + ϵ)
ϵ∼clip(𝒩(0, σ), -c, c )
where the added noise is clipped to keep the target close to the original action. The outcome is an algorithm reminiscent of Expected SARSA, where the value estimate is instead learned off-policy and the noise added to the target policy is chosen independently of the exploration policy. The value estimate learned is with respect to a noisy policy defined by the parameter σ.
In Syntology for a different name, 'policy smoothing', about another paper, arXiv:2106.11420: verified reference implementation at level V3; retrieve with get_reference_implementation(name='policy smoothing'). The lookup matched a different name in Syntology's graph, 'policy smoothing' (resolution: token_match); this answer is about that name. About another paper: this answer is about arXiv:2106.11420, while the archive credits arXiv:1802.09477.
Origin paper of this answer, arXiv:2106.11420 (its page on Syntology).
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 116 · with a code link: 45 · where Syntology ran a sample: 18 (17 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (18 of 116 tagged: 17 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 116, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning 6 Jun 2025 · 1 repository · arXiv:2506.05968Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples)
-
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control 28 May 2025 · 0 repositories · arXiv:2505.22642
-
LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models 21 May 2025 · 0 repositories · arXiv:2505.15293
-
Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Control 13 May 2025 · 0 repositories · arXiv:2505.09029
-
Energy Efficient RSMA-Based LEO Satellite Communications Assisted by UAV-Mounted BD-Active RIS: A DRL Approach 7 May 2025 · 0 repositories · arXiv:2505.04148
-
AlphaGrad: Non-Linear Gradient Normalization Optimizer 22 Apr 2025 · 0 repositories · arXiv:2504.16020
-
Motion Control in Multi-Rotor Aerial Robots Using Deep Reinforcement Learning 9 Feb 2025 · 0 repositories · arXiv:2502.05996
-
TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation 5 Feb 2025 · 1 repository · arXiv:2502.02854
-
EvoRL: A GPU-accelerated Framework for Evolutionary Reinforcement Learning 25 Jan 2025 · 2 repositories · arXiv:2501.15129
-
Enhancing UAV Path Planning Efficiency Through Accelerated Learning 17 Jan 2025 · 0 repositories · arXiv:2501.10141
-
An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space 17 Dec 2024 · 0 repositories · arXiv:2412.12605
-
Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning 20 Nov 2024 · 0 repositories · arXiv:2411.13116
-
Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers 31 Oct 2024 · 1 repository · arXiv:2410.24108Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
NetworkGym: Reinforcement Learning Environments for Multi-Access Traffic Management in Network Simulation 30 Oct 2024 · 1 repository · arXiv:2411.04138Syntology official (archive's flag): 6 ran · 15 ran (of which 2 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 0 violated, 3 with no contract checked; 8 where Syntology's instrument failed) · 3 unverified (of 18 harvested samples) · 1 pointer-only (licence)
-
Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution 29 Oct 2024 · 1 repository · arXiv:2410.21940
-
Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions 15 Oct 2024 · 0 repositories · arXiv:2410.11833
-
Navigation in a simplified Urban Flow through Deep Reinforcement Learning 26 Sep 2024 · 0 repositories · arXiv:2409.17922
-
Simultaneous Training of First- and Second-Order Optimizers in Population-Based Reinforcement Learning 27 Aug 2024 · 0 repositories · arXiv:2408.15421
-
Optimizing TD3 for 7-DOF Robotic Arm Grasping: Overcoming Suboptimality with Exploration-Enhanced Contrastive Learning 26 Aug 2024 · 0 repositories · arXiv:2408.14009
-
Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks 31 Jul 2024 · 0 repositories · arXiv:2407.21338
-
Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity 29 Jun 2024 · 0 repositories · arXiv:2407.09557
-
Performance Comparison of Deep RL Algorithms for Mixed Traffic Cooperative Lane-Changing 25 Jun 2024 · 0 repositories · arXiv:2407.02521
-
Value Improved Actor Critic Algorithms 3 Jun 2024 · 0 repositories · arXiv:2406.01423
-
Chaos-based reinforcement learning with TD3 15 May 2024 · 0 repositories · arXiv:2405.09086
-
Offline Reinforcement Learning with Behavioral Supervisor Tuning 25 Apr 2024 · 0 repositories · arXiv:2404.16399
-
RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Models 11 Mar 2024 · 0 repositories · arXiv:2403.06420
-
Conservative DDPG -- Pessimistic RL without Ensemble 8 Mar 2024 · 0 repositories · arXiv:2403.05732
-
Fill-and-Spill: Deep Reinforcement Learning Policy Gradient Methods for Reservoir Operation Decision and Control 7 Mar 2024 · 0 repositories · arXiv:2403.04195
-
Exploiting Estimation Bias in Clipped Double Q-Learning for Continous Control Reinforcement Learning Tasks 14 Feb 2024 · 0 repositories · arXiv:2402.09078
-
MinMaxMin Q-learning 3 Feb 2024 · 0 repositories · arXiv:2402.05951
-
SQT -- std Q-target 3 Feb 2024 · 0 repositories · arXiv:2402.05950
-
Neural Style Transfer with Twin-Delayed DDPG for Shared Control of Robotic Manipulators 1 Feb 2024 · 0 repositories · arXiv:2402.00722
-
Reconciling Spatial and Temporal Abstractions for Goal Representation 18 Jan 2024 · 1 repository · arXiv:2401.09870Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 6 unverified (of 11 harvested samples) · 11 pointer-only (licence)
-
Sum Throughput Maximization in Multi-BD Symbiotic Radio NOMA Network Assisted by Active-STAR-RIS 16 Jan 2024 · 0 repositories · arXiv:2401.08301
-
An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control 11 Jan 2024 · 1 repository · arXiv:2401.05737
-
Adaptive Kalman-based hybrid car following strategy using TD3 and CACC 26 Dec 2023 · 0 repositories · arXiv:2312.15993
-
Multiagent Copilot Approach for Shared Autonomy between Human EEG and TD3 Deep Reinforcement Learning 22 Dec 2023 · 0 repositories · arXiv:2312.14458
-
6G Fresnel Spot Beamfocusing using Large-Scale Metasurfaces: A Distributed DRL-Based Approach 18 Nov 2023 · 0 repositories · arXiv:2311.11109
-
Mitigating Estimation Errors by Twin TD-Regularized Actor and Critic for Deep Reinforcement Learning 7 Nov 2023 · 0 repositories · arXiv:2311.03711
-
On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economics 30 Oct 2023 · 0 repositories · arXiv:2310.19527
-
Hierarchical Framework for Interpretable and Probabilistic Model-Based Safe Reinforcement Learning 28 Oct 2023 · 1 repository · arXiv:2310.18811
-
Distributional Soft Actor-Critic with Three Refinements 9 Oct 2023 · 2 repositories · arXiv:2310.05858Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
A Long N-step Surrogate Stage Reward for Deep Reinforcement Learning 21 Sep 2023 · 0 repositories
-
Double Gumbel Q-Learning 21 Sep 2023 · 1 repository
-
Taylor TD-learning 21 Sep 2023 · 0 repositories
-
Caching-at-STARS: the Next Generation Edge Caching 1 Aug 2023 · 0 repositories · arXiv:2308.00562
-
Evolutionary Strategy Guided Reinforcement Learning via MultiBuffer Communication 20 Jun 2023 · 0 repositories · arXiv:2306.11535
-
RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control 6 Jun 2023 · 1 repository · arXiv:2306.03530
-
For SALE: State-Action Representation Learning for Deep Reinforcement Learning 4 Jun 2023 · 2 repositories · arXiv:2306.02451Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 1 pointer-only (licence)
-
ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages 2 Jun 2023 · 1 repository · arXiv:2306.01460
-
Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architectures 30 May 2023 · 1 repository · arXiv:2305.18701
-
Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization 31 Mar 2023 · 0 repositories · arXiv:2304.00026
-
A Strategy-Oriented Bayesian Soft Actor-Critic Model 7 Mar 2023 · 0 repositories · arXiv:2303.04193
-
Swim: A General-Purpose, High-Performing, and Efficient Activation Function for Locomotion Control Tasks 5 Mar 2023 · 1 repository · arXiv:2303.02640
-
The Ladder in Chaos: A Simple and Effective Improvement to General DRL Algorithms by Policy Path Trimming and Boosting 2 Mar 2023 · 0 repositories · arXiv:2303.01391
-
Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning 13 Feb 2023 · 1 repository · arXiv:2302.06548Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 1 pointer-only (licence)
-
RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework 10 Feb 2023 · 0 repositories · arXiv:2302.04994
-
Automated deep reinforcement learning for real-time scheduling strategy of multi-energy system integrated with post-carbon and direct-air carbon captured system 18 Jan 2023 · 0 repositories · arXiv:2301.07768
-
Actor-Director-Critic: A Novel Deep Reinforcement Learning Framework 10 Jan 2023 · 0 repositories · arXiv:2301.03887
-
Extreme Q-Learning: MaxEnt RL without Entropy 5 Jan 2023 · 4 repositories · arXiv:2301.02328Syntology 8 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified (of 13 harvested samples) · 7 pointer-only (licence)
-
Off-Policy Deep Reinforcement Learning Algorithms for Handling Various Robotic Manipulator Tasks 11 Dec 2022 · 0 repositories · arXiv:2212.05572
-
TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets 5 Dec 2022 · 1 repository · arXiv:2212.02125
-
Algorithmic Trading Using Continuous Action Space Deep Reinforcement Learning 7 Oct 2022 · 0 repositories · arXiv:2210.03469
-
A Novel Entropy-Maximizing TD3-based Reinforcement Learning for Automatic PID Tuning 5 Oct 2022 · 0 repositories · arXiv:2210.02381
-
Experimental Study on The Effect of Multi-step Deep Reinforcement Learning in POMDPs 12 Sep 2022 · 1 repository · arXiv:2209.04999
-
Bayesian Soft Actor-Critic: A Directed Acyclic Strategy Graph Based Deep Reinforcement Learning 11 Aug 2022 · 2 repositories · arXiv:2208.06033
-
Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Scheduling 1 Aug 2022 · 1 repository · arXiv:2208.00728
-
Multi-Agent Deep Reinforcement Learning-Driven Mitigation of Adverse Effects of Cyber-Attacks on Electric Vehicle Charging Station 14 Jul 2022 · 0 repositories · arXiv:2207.07041
-
Equivariant Reinforcement Learning for Quadrotor UAV 2 Jun 2022 · 0 repositories · arXiv:2206.01233
-
Non-Parametric Stochastic Policy Gradient with Strategic Retreat for Non-Stationary Environment 24 Mar 2022 · 0 repositories · arXiv:2203.14905
-
Adaptively Calibrated Critic Estimates for Deep Reinforcement Learning 24 Nov 2021 · 1 repository · arXiv:2111.12673Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
Improving Learning from Demonstrations by Learning from Experience 16 Nov 2021 · 0 repositories · arXiv:2111.08156
-
Recurrent Off-policy Baselines for Memory-based Continuous Control 25 Oct 2021 · 1 repository · arXiv:2110.12628
-
Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations 3 Oct 2021 · 0 repositories · arXiv:2110.01101
-
Explanation-Aware Experience Replay in Rule-Dense Environments 29 Sep 2021 · 1 repository · arXiv:2109.14711
-
Faster Reinforcement Learning with Value Target Lower Bounding 29 Sep 2021 · 0 repositories
-
Meta Attention For Off-Policy Actor-Critic 29 Sep 2021 · 0 repositories
-
Robust and Data-efficient Q-learning by Composite Value-estimation 29 Sep 2021 · 0 repositories
-
SPP-RL: State Planning Policy Reinforcement Learning 29 Sep 2021 · 0 repositories
-
ADER:Adapting between Exploration and Robustness for Actor-Critic Methods 8 Sep 2021 · 0 repositories · arXiv:2109.03443
-
WAD: A Deep Reinforcement Learning Agent for Urban Autonomous Driving 27 Aug 2021 · 0 repositories · arXiv:2108.12134
-
Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study 5 Aug 2021 · 0 repositories · arXiv:2108.02665
-
Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings 28 Jul 2021 · 0 repositories · arXiv:2107.13356
-
Efficient Continuous Control with Double Actors and Regularized Critics 6 Jun 2021 · 1 repository · arXiv:2106.03050
-
Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition 9 Mar 2021 · 1 repository · arXiv:2103.05174
-
Twin actor twin delayed deep deterministic policy gradient (TATD3) learning for batch process control 25 Feb 2021 · 0 repositories · arXiv:2102.13012
-
Memory-based Deep Reinforcement Learning for POMDPs 24 Feb 2021 · 1 repository · arXiv:2102.12344Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
OffCon³: What is state of the art anyway? 27 Jan 2021 · 1 repository · arXiv:2101.11331
-
GST: Group-Sparse Training for Accelerating Deep Reinforcement Learning 24 Jan 2021 · 0 repositories · arXiv:2101.09650
-
Learning Synthetic Environments for Reinforcement Learning with Evolution Strategies 24 Jan 2021 · 1 repository · arXiv:2101.09721
-
PGPS : Coupling Policy Gradient with Population-based Search 1 Jan 2021 · 0 repositories
-
Policy Gradient RL Algorithms as Directed Acyclic Graphs 14 Dec 2020 · 1 repository · arXiv:2012.07763
-
Ranking Items in Large-Scale Item Search Engines with Reinforcement Learning 14 Dec 2020 · 0 repositories
-
Virtual Autonomous Driving with Reinforcement Learning 14 Dec 2020 · 0 repositories
-
OPAC: Opportunistic Actor-Critic 11 Dec 2020 · 0 repositories · arXiv:2012.06555
-
Efficient Reservoir Management through Deep Reinforcement Learning 7 Dec 2020 · 0 repositories · arXiv:2012.03822
-
FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance 19 Nov 2020 · 6 repositories · arXiv:2011.09607
-
Tonic: A Deep Reinforcement Learning Library for Fast Prototyping and Benchmarking 15 Nov 2020 · 1 repository · arXiv:2011.07537Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
RealAnt: An Open-Source Low-Cost Quadruped for Education and Research in Real-World Reinforcement Learning 5 Nov 2020 · 1 repository · arXiv:2011.03085
-
Hindsight Experience Replay with Kronecker Product Approximate Curvature 9 Oct 2020 · 0 repositories · arXiv:2010.06142
Tasks archive 2025-07-28
20 shown of 69 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections