Methods › Reinforcement Learning › Off-Policy TD Control › Double Q-learning
Double Q-learning
Introduced by Hado V. Hasselt in Double Q-learning
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Double Q-learning is an off-policy reinforcement learning algorithm that utilises double estimation to counteract overestimation problems with traditional Q-learning.
The max operator in standard Q-learning and DQN uses the same values both to select and to evaluate an action. This makes it more likely to select overestimated values, resulting in overoptimistic value estimates. To prevent this, we can decouple the selection from the evaluation, which is the idea behind Double Q-learning:
Y^Qₜ = Rₜ₊₁ + γQ(Sₜ₊₁, maxₐQ(Sₜ₊₁, a; θₜ);θₜ)
The Double Q-learning error can then be written as:
Y^(DoubleQ)ₜ = Rₜ₊₁ + γQ(Sₜ₊₁, maxₐQ(Sₜ₊₁, a; θₜ);θ^′ₜ)
Here the selection of the action in the max is still due to the online weights θₜ. But we use a second set of weights θ^′ₜ to fairly evaluate the value of this policy.
Papers archive 2025-07-28
30 shown of 112, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Reinforcement Learning-Based Policy Optimisation For Heterogeneous Radio Access 18 Jun 2025 · 0 repositories · arXiv:2506.15273
-
Dynamic Operating System Scheduling Using Double DQN: A Reinforcement Learning Approach to Task Optimization 31 Mar 2025 · 0 repositories · arXiv:2503.23659
-
Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework 17 Dec 2024 · 0 repositories · arXiv:2412.12597
-
Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC 6 Nov 2024 · 3 repositories · arXiv:2411.03820Syntology ran 13 of 25 samples · 12 unverified · 21 pointer-only (licence)
-
Bootstrapping Expectiles in Reinforcement Learning 6 Jun 2024 · 0 repositories · arXiv:2406.04081
-
A New View on Planning in Online Reinforcement Learning 3 Jun 2024 · 0 repositories · arXiv:2406.01562
-
Active search and coverage using point-cloud reinforcement learning 18 Dec 2023 · 0 repositories · arXiv:2312.11410
-
Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization 10 Dec 2023 · 1 repository · arXiv:2312.05787
-
Data-efficient Deep Reinforcement Learning for Vehicle Trajectory Control 30 Nov 2023 · 0 repositories · arXiv:2311.18393
-
Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models 9 Nov 2023 · 0 repositories · arXiv:2311.05743
-
Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets 25 Sep 2023 · 0 repositories · arXiv:2309.13955
-
Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions 20 Sep 2023 · 0 repositories · arXiv:2309.10980
-
Deep Reinforcement Learning for Artificial Upwelling Energy Management 20 Aug 2023 · 0 repositories · arXiv:2308.10199
-
Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication 5 Jul 2023 · 0 repositories · arXiv:2307.02002
-
Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning 27 Jun 2023 · 0 repositories · arXiv:2306.15585
-
Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm 17 Jun 2023 · 0 repositories · arXiv:2306.10216
-
RSRM: Reinforcement Symbolic Regression Machine 24 May 2023 · 0 repositories · arXiv:2305.14656
-
Extracting Diagnosis Pathways from Electronic Health Records Using Deep Reinforcement Learning 10 May 2023 · 1 repository · arXiv:2305.06295
-
Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity 7 May 2023 · 1 repository · arXiv:2305.04180
-
Smoothed Q-learning 15 Mar 2023 · 0 repositories · arXiv:2303.08631
-
Double A3C: Deep Reinforcement Learning on OpenAI Gym Games 4 Mar 2023 · 0 repositories · arXiv:2303.02271
-
Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets 20 Jan 2023 · 0 repositories · arXiv:2301.08688
-
Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity 13 Jan 2023 · 0 repositories · arXiv:2301.05391
-
Neighboring state-based RL Exploration 21 Dec 2022 · 0 repositories · arXiv:2212.10712
-
Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks 7 Oct 2022 · 0 repositories · arXiv:2210.03325
-
M²DQN: A Robust Method for Accelerating Deep Q-learning Network 16 Sep 2022 · 1 repository · arXiv:2209.07809
-
Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks 16 Sep 2022 · 1 repository · arXiv:2209.07670
-
Double Q-Learning for Citizen Relocation During Natural Hazards 8 Sep 2022 · 0 repositories · arXiv:2209.03800
-
Deep Reinforcement Learning with Swin Transformers 30 Jun 2022 · 1 repository · arXiv:2206.15269
-
DNA: Proximal Policy Optimization with a Dual Network Architecture 20 Jun 2022 · 1 repository · arXiv:2206.10027
Tasks archive 2025-07-28
20 shown of 67 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections