</> Code with Papers by Syntology

Methods › General › Regularization › Target Policy Smoothing

Target Policy Smoothing

116 papers tagged archive 2025-07-28

Introduced by Scott Fujimoto et al. in Addressing Function Approximation Error in Actor-Critic Methods

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Target Policy Smoothing is a regularization strategy for the value function in reinforcement learning. Deterministic policies can overfit to narrow peaks in the value estimate, making them highly susceptible to functional approximation error, increasing the variance of the target. To reduce this variance, target policy smoothing adds a small amount of random noise to the target policy and averages over mini-batches - approximating a SARSA-like expectation/integral.

The modified target update is:

y = r + γQ_(θ′)(s′, π_(θ′)(s′) + ϵ)

ϵ∼clip(𝒩(0, σ), -c, c )

where the added noise is clipped to keep the target close to the original action. The outcome is an algorithm reminiscent of Expected SARSA, where the value estimate is instead learned off-policy and the noise added to the target policy is chosen independently of the exploration policy. The value estimate learned is with respect to a noisy policy defined by the parameter σ.

PaperSourceSee Code · sfujim/TD3

Papers archive 2025-07-28

30 shown of 116, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Tasks archive 2025-07-28

20 shown of 69 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Reinforcement Learning (RL)67
reinforcement-learning54
Deep Reinforcement Learning53
Reinforcement Learning46
Continuous Control30
continuous-control29
MuJoCo25
Q-Learning11
OpenAI Gym9
Decision Making8
Autonomous Driving6
Offline RL5
Benchmarking4
Meta-Learning4
Policy Gradient Methods4
D4RL3
GPU3
Scheduling3
Atari Games2
Computational Efficiency2

Usage over time archive 2025-07-28

Papers per year tagged with Target Policy Smoothing: 2018 to 2025, peak 25 25 0 2018: 1 paper 2018 2019: 6 papers 2019 2020: 18 papers 2020 2021: 21 papers 2021 2022: 10 papers 2022 2023: 25 papers 2023 2024: 25 papers 2024 2025: 10 papers 2025
Papers per year the archive tags with this method, by the paper's archive date (116 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Regularization

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections