{"url":"/method/double-dqn","slug":"double-dqn","name":"Double DQN","full_name":"Double DQN","full_name_withheld":false,"description_markdown":"A **Double Deep Q-Network**, or **Double DQN** utilises [Double Q-learning](https://paperswithcode.com/method/double-q-learning) to reduce overestimation by decomposing the max operation in the target into action selection and action evaluation. We evaluate the greedy policy according to the online network, but we use the target network to estimate its value.  The update is the same as for [DQN](https://paperswithcode.com/method/dqn), but replacing the target $Y^{DQN}\\_{t}$ with:\r\n\r\n$$ Y^{DoubleDQN}\\_{t} = R\\_{t+1}+\\gamma{Q}\\left(S\\_{t+1}, \\arg\\max\\_{a}Q\\left(S\\_{t+1}, a; \\theta\\_{t}\\right);\\theta\\_{t}^{-}\\right) $$\r\n\r\nCompared to the original formulation of Double [Q-Learning](https://paperswithcode.com/method/q-learning), in Double DQN the weights of the second network $\\theta^{'}\\_{t}$ are replaced with the weights of the target network $\\theta\\_{t}^{-}$ for the evaluation of the current greedy policy.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1509.06461v3","title":"Deep Reinforcement Learning with Double Q-learning","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Q-Learning Networks","url":"/methods/category/q-learning-networks","pwc_aliases":["q-learning"]}],"n_papers_tagged":45,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"Dynamic Operating System Scheduling Using Double DQN: A Reinforcement Learning Approach to Task Optimization","date":"2025-03-31","arxiv_id":"2503.23659","n_code_links":0,"syntology":null},{"paper":null,"title":"Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework","date":"2024-12-17","arxiv_id":"2412.12597","n_code_links":0,"syntology":null},{"paper":null,"title":"A New View on Planning in Online Reinforcement Learning","date":"2024-06-03","arxiv_id":"2406.01562","n_code_links":0,"syntology":null},{"paper":null,"title":"Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models","date":"2023-11-09","arxiv_id":"2311.05743","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets","date":"2023-09-25","arxiv_id":"2309.13955","n_code_links":0,"syntology":null},{"paper":null,"title":"Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions","date":"2023-09-20","arxiv_id":"2309.10980","n_code_links":0,"syntology":null},{"paper":null,"title":"Interpretable and Secure Trajectory Optimization for UAV-Assisted Communication","date":"2023-07-05","arxiv_id":"2307.02002","n_code_links":0,"syntology":null},{"paper":null,"title":"Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm","date":"2023-06-17","arxiv_id":"2306.10216","n_code_links":0,"syntology":null},{"paper":"/paper/extracting-diagnosis-pathways-from-electronic","title":"Extracting Diagnosis Pathways from Electronic Health Records Using Deep Reinforcement Learning","date":"2023-05-10","arxiv_id":"2305.06295","n_code_links":1,"syntology":null},{"paper":"/paper/train-a-real-world-local-path-planner-in-one","title":"Train a Real-world Local Path Planner in One Hour via Partially Decoupled Reinforcement Learning and Vectorized Diversity","date":"2023-05-07","arxiv_id":"2305.04180","n_code_links":1,"syntology":null},{"paper":null,"title":"Neighboring state-based RL Exploration","date":"2022-12-21","arxiv_id":"2212.10712","n_code_links":0,"syntology":null},{"paper":null,"title":"Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks","date":"2022-10-07","arxiv_id":"2210.03325","n_code_links":0,"syntology":null},{"paper":"/paper/m-2-dqn-a-robust-method-for-accelerating-deep","title":"M$^2$DQN: A Robust Method for Accelerating Deep Q-learning Network","date":"2022-09-16","arxiv_id":"2209.07809","n_code_links":1,"syntology":null},{"paper":"/paper/deep-reinforcement-learning-with-swin","title":"Deep Reinforcement Learning with Swin Transformers","date":"2022-06-30","arxiv_id":"2206.15269","n_code_links":1,"syntology":null},{"paper":null,"title":"Goal-Space Planning with Subgoal Models","date":"2022-06-06","arxiv_id":"2206.02902","n_code_links":0,"syntology":null},{"paper":null,"title":"Random Ensemble Reinforcement Learning for Traffic Signal Control","date":"2022-03-10","arxiv_id":"2203.05961","n_code_links":0,"syntology":null},{"paper":"/paper/skrl-modular-and-flexible-library-for","title":"skrl: Modular and Flexible Library for Reinforcement Learning","date":"2022-02-08","arxiv_id":"2202.03825","n_code_links":1,"syntology":null},{"paper":null,"title":"An Improved Reinforcement Learning Algorithm for Learning to Branch","date":"2022-01-17","arxiv_id":"2201.06213","n_code_links":0,"syntology":null},{"paper":"/paper/intelligent-traffic-light-via-policy-based","title":"Intelligent Traffic Light via Policy-based Deep Reinforcement Learning","date":"2021-12-27","arxiv_id":"2112.13817","n_code_links":1,"syntology":null},{"paper":null,"title":"Modified Double DQN: addressing stability","date":"2021-08-09","arxiv_id":"2108.04115","n_code_links":0,"syntology":null},{"paper":null,"title":"An Improved Algorithm of Robot Path Planning in Complex Environment Based on Double DQN","date":"2021-07-23","arxiv_id":"2107.11245","n_code_links":0,"syntology":null},{"paper":"/paper/a-reinforcement-learning-environment-for-2","title":"A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis","date":"2021-07-15","arxiv_id":"2107.07373","n_code_links":1,"syntology":null},{"paper":null,"title":"Learning to Optimize Industry-Scale Dynamic Pickup and Delivery Problems","date":"2021-05-27","arxiv_id":"2105.12899","n_code_links":0,"syntology":null},{"paper":"/paper/independent-reinforcement-learning-for-weakly","title":"Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem","date":"2021-04-22","arxiv_id":"2104.10917","n_code_links":1,"syntology":null},{"paper":null,"title":"Greedy-Step Off-Policy Reinforcement Learning","date":"2021-02-23","arxiv_id":"2102.11717","n_code_links":0,"syntology":null},{"paper":"/paper/recurrent-rational-networks","title":"Adaptive Rational Activations to Boost Deep Reinforcement Learning","date":"2021-02-18","arxiv_id":"2102.09407","n_code_links":4,"syntology":{"ran":6,"of":6,"unverified":0,"pointer_only":3}},{"paper":null,"title":"Mobile Robots Autonomous Exploration with Reinforcement Learning","date":"2020-12-14","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Self-correcting Q-Learning","date":"2020-12-02","arxiv_id":"2012.01100","n_code_links":0,"syntology":null},{"paper":null,"title":"Chrome Dino Run using Reinforcement Learning","date":"2020-08-15","arxiv_id":"2008.06799","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-learning-with-quantum","title":"Reinforcement Learning with Quantum Variational Circuits","date":"2020-08-15","arxiv_id":"2008.07524","n_code_links":2,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":27},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":26},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":25},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":16},{"task":"/task/q-learning","name":"Q-Learning","papers":15},{"task":"/task/atari-games","name":"Atari Games","papers":8},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":5},{"task":"/task/decision-making","name":"Decision Making","papers":3},{"task":"/task/general-reinforcement-learning","name":"General Reinforcement Learning","papers":2},{"task":"/task/model-based-reinforcement-learning","name":"Model-based Reinforcement Learning","papers":2},{"task":"/task/scheduling","name":"Scheduling","papers":2},{"task":"/task/traffic-signal-control","name":"Traffic Signal Control","papers":2},{"task":"/task/algorithmic-trading","name":"Algorithmic Trading","papers":1},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":1},{"task":"/task/cloud-computing","name":"Cloud Computing","papers":1},{"task":"/task/collision-avoidance","name":"Collision Avoidance","papers":1},{"task":"/task/combinatorial-optimization","name":"Combinatorial Optimization","papers":1},{"task":"/task/conformal-prediction","name":"Conformal Prediction","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1},{"task":"/task/efficient-exploration","name":"Efficient Exploration","papers":1}],"tasks_shown":20,"n_tasks":43,"usage_by_year":[{"year":"2015","papers":1},{"year":"2018","papers":4},{"year":"2019","papers":4},{"year":"2020","papers":10},{"year":"2021","papers":8},{"year":"2022","papers":8},{"year":"2023","papers":7},{"year":"2024","papers":2},{"year":"2025","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/double-dqn"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}