{"url":"/method/q-learning","slug":"q-learning","name":"Q-Learning","full_name":"Q-Learning","full_name_withheld":false,"description_markdown":"**Q-Learning** is an off-policy temporal difference control algorithm:\r\n\r\n$$Q\\left(S\\_{t}, A\\_{t}\\right) \\leftarrow Q\\left(S\\_{t}, A\\_{t}\\right) + \\alpha\\left[R_{t+1} + \\gamma\\max\\_{a}Q\\left(S\\_{t+1}, a\\right) - Q\\left(S\\_{t}, A\\_{t}\\right)\\right] $$\r\n\r\nThe learned action-value function $Q$ directly approximates $q\\_{*}$, the optimal action-value function, independent of the policy being followed.\r\n\r\nSource: Sutton and Barto, Reinforcement Learning, 2nd Edition","description_state":"present","introduced_year":1984,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Off-Policy TD Control","url":"/methods/category/off-policy-td-control","pwc_aliases":[]}],"n_papers_tagged":1734,"archive_num_papers":1734,"papers_newest_first":[{"paper":null,"title":"Detecting and Mitigating Reward Hacking in Reinforcement Learning Systems: A Comprehensive Empirical Study","date":"2025-07-08","arxiv_id":"2507.05619","n_code_links":0,"syntology":null},{"paper":null,"title":"2048: Reinforcement Learning in a Delayed Reward Environment","date":"2025-07-07","arxiv_id":"2507.05465","n_code_links":0,"syntology":null},{"paper":null,"title":"VRAIL: Vectorized Reward-based Attribution for Interpretable Learning","date":"2025-06-19","arxiv_id":"2506.16014","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning-Based Policy Optimisation For Heterogeneous Radio Access","date":"2025-06-18","arxiv_id":"2506.15273","n_code_links":0,"syntology":null},{"paper":null,"title":"GCN-Driven Reinforcement Learning for Probabilistic Real-Time Guarantees in Industrial URLLC","date":"2025-06-17","arxiv_id":"2506.15011","n_code_links":0,"syntology":null},{"paper":null,"title":"Implicit Constraint-Aware Off-Policy Correction for Offline Reinforcement Learning","date":"2025-06-16","arxiv_id":"2506.14058","n_code_links":0,"syntology":null},{"paper":null,"title":"ReinDSplit: Reinforced Dynamic Split Learning for Pest Recognition in Precision Agriculture","date":"2025-06-16","arxiv_id":"2506.13935","n_code_links":0,"syntology":null},{"paper":null,"title":"\"What are my options?\": Explaining RL Agents with Diverse Near-Optimal Alternatives (Extended)","date":"2025-06-11","arxiv_id":"2506.09901","n_code_links":0,"syntology":null},{"paper":null,"title":"Reliable Critics: Monotonic Improvement and Convergence Guarantees for Reinforcement Learning","date":"2025-06-08","arxiv_id":"2506.07134","n_code_links":0,"syntology":null},{"paper":null,"title":"Bridging the Performance Gap Between Target-Free and Target-Based Reinforcement Learning With Iterated Q-Learning","date":"2025-06-04","arxiv_id":"2506.04398","n_code_links":0,"syntology":null},{"paper":null,"title":"Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons","date":"2025-06-03","arxiv_id":"2506.03392","n_code_links":0,"syntology":null},{"paper":null,"title":"Getting More from Less: Transfer Learning Improves Sleep Stage Decoding Accuracy in Peripheral Wearable Devices","date":"2025-05-31","arxiv_id":"2506.00730","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning for Hanabi","date":"2025-05-31","arxiv_id":"2506.00458","n_code_links":0,"syntology":null},{"paper":null,"title":"Combining Deep Architectures for Information Gain estimation and Reinforcement Learning for multiagent field exploration","date":"2025-05-29","arxiv_id":"2505.23865","n_code_links":0,"syntology":null},{"paper":null,"title":"On Global Convergence Rates for Federated Policy Gradient under Heterogeneous Environment","date":"2025-05-29","arxiv_id":"2505.23459","n_code_links":0,"syntology":null},{"paper":null,"title":"BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL","date":"2025-05-28","arxiv_id":"2505.21974","n_code_links":0,"syntology":null},{"paper":null,"title":"A General-Purpose Theorem for High-Probability Bounds of Stochastic Approximation with Polyak Averaging","date":"2025-05-27","arxiv_id":"2505.21796","n_code_links":0,"syntology":null},{"paper":"/paper/the-cell-must-go-on-agar-io-for-continual","title":"The Cell Must Go On: Agar.io for Continual Reinforcement Learning","date":"2025-05-23","arxiv_id":"2505.18347","n_code_links":1,"syntology":null},{"paper":null,"title":"Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies","date":"2025-05-22","arxiv_id":"2505.16242","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning for Stock Transactions","date":"2025-05-22","arxiv_id":"2505.16099","n_code_links":0,"syntology":null},{"paper":null,"title":"LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models","date":"2025-05-21","arxiv_id":"2505.15293","n_code_links":0,"syntology":null},{"paper":null,"title":"OPA-Pack: Object-Property-Aware Robotic Bin Packing","date":"2025-05-19","arxiv_id":"2505.13339","n_code_links":0,"syntology":null},{"paper":null,"title":"When a Reinforcement Learning Agent Encounters Unknown Unknowns","date":"2025-05-19","arxiv_id":"2505.13188","n_code_links":0,"syntology":null},{"paper":null,"title":"Imagination-Limited Q-Learning for Offline Reinforcement Learning","date":"2025-05-18","arxiv_id":"2505.12211","n_code_links":0,"syntology":null},{"paper":null,"title":"ShiQ: Bringing back Bellman to LLMs","date":"2025-05-16","arxiv_id":"2505.11081","n_code_links":0,"syntology":null},{"paper":null,"title":"Automatic Reward Shaping from Confounded Offline Data","date":"2025-05-16","arxiv_id":"2505.11478","n_code_links":0,"syntology":null},{"paper":null,"title":"Bias or Optimality? Disentangling Bayesian Inference and Learning Biases in Human Decision-Making","date":"2025-05-12","arxiv_id":"2505.08049","n_code_links":0,"syntology":null},{"paper":null,"title":"Convert Language Model into a Value-based Strategic Planner","date":"2025-05-11","arxiv_id":"2505.06987","n_code_links":0,"syntology":null},{"paper":null,"title":"A Large Language Model-Enhanced Q-learning for Capacitated Vehicle Routing Problem with Time Windows","date":"2025-05-09","arxiv_id":"2505.06178","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning for Game-Theoretic Resource Allocation on Graphs","date":"2025-05-08","arxiv_id":"2505.06319","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/q-learning","name":"Q-Learning","papers":1321},{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":898},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":783},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":776},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":409},{"task":"/task/decision-making","name":"Decision Making","papers":134},{"task":"/task/atari-games","name":"Atari Games","papers":103},{"task":"/task/multi-agent-reinforcement-learning","name":"Multi-agent Reinforcement Learning","papers":100},{"task":"/task/management","name":"Management","papers":74},{"task":"/task/offline-rl","name":"Offline RL","papers":64},{"task":"/task/scheduling","name":"Scheduling","papers":60},{"task":"/task/continuous-control","name":"Continuous Control","papers":46},{"task":"/task/openai-gym","name":"OpenAI Gym","papers":45},{"task":"/task/continuous-control","name":"continuous-control","papers":42},{"task":"/task/sequential-decision-making","name":"Sequential Decision Making","papers":34},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":33},{"task":"/task/imitation-learning","name":"Imitation Learning","papers":31},{"task":"/task/d4rl","name":"D4RL","papers":28},{"task":"/task/mujoco","name":"MuJoCo","papers":27},{"task":"/task/autonomous-vehicles","name":"Autonomous Vehicles","papers":25}],"tasks_shown":20,"n_tasks":393,"usage_by_year":[{"year":"2009","papers":1},{"year":"2010","papers":1},{"year":"2011","papers":1},{"year":"2013","papers":6},{"year":"2014","papers":5},{"year":"2015","papers":17},{"year":"2016","papers":23},{"year":"2017","papers":64},{"year":"2018","papers":110},{"year":"2019","papers":181},{"year":"2020","papers":243},{"year":"2021","papers":269},{"year":"2022","papers":188},{"year":"2023","papers":250},{"year":"2024","papers":269},{"year":"2025","papers":106}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/q-learning"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}