{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/human-level-control-through-deep","title":"Human level control through deep reinforcement learning","arxiv_id":null,"date":"2015-02-25","proceeding":null,"authors":["Volodymyr Mnih","Koray Kavukcuoglu","David Silver","Andrei A. Rusu","Joel Veness","Marc G. Bellemare","Alex Graves","Martin Riedmiller","Andreas K. Fidjeland","Georg Ostrovski","Stig Petersen","Charles Beattie","Amir Sadik","Ioannis Antonoglou","Helen King","Dharshan Kumaran","Daan Wierstra","Shane Legg1 & Demis Hassabis"],"abstract":"The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.","url_abs":"https://www.nature.com/articles/nature14236","url_pdf":"https://www.nature.com/articles/nature14236.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/2023-MindSpore-1/ms-code-52","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok"}},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/MaximeVandegar/Papers-in-100-Lines-of-Code/tree/main/Deep_Reinforcement_Learning_with_Double_Q_learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/epignatelli/human-level-control-through-deep-reinforcement-learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/gordicaleksa/pytorch-learn-reinforcement-learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/grass123-hub/DQN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok"}},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/mindspore-courses/Rainbow-MindSpore","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/mindspore-courses/Rainbow-MindSpore/blob/main/01.dqn.ipynb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"human-level-control-through-deep","repo_url":"https://github.com/xiuyu0000/new_papers_codes/tree/main/dqn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/atari-games-on-atari-2600-alien","task":"Atari Games","dataset":"Atari 2600 Alien","model":"Nature DQN","rank_in_archive_order":27,"of":49,"metrics":{"Score":"3069.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-amidar","task":"Atari Games","dataset":"Atari 2600 Amidar","model":"Nature DQN","rank_in_archive_order":28,"of":48,"metrics":{"Score":"739.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-assault","task":"Atari Games","dataset":"Atari 2600 Assault","model":"Nature DQN","rank_in_archive_order":35,"of":45,"metrics":{"Score":"3359.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-asterix","task":"Atari Games","dataset":"Atari 2600 Asterix","model":"Nature DQN","rank_in_archive_order":39,"of":49,"metrics":{"Score":"6012"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-asteroids","task":"Atari Games","dataset":"Atari 2600 Asteroids","model":"Nature DQN","rank_in_archive_order":31,"of":43,"metrics":{"Score":"1629.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-atlantis","task":"Atari Games","dataset":"Atari 2600 Atlantis","model":"Nature DQN","rank_in_archive_order":40,"of":42,"metrics":{"Score":"85641.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-bank-heist","task":"Atari Games","dataset":"Atari 2600 Bank Heist","model":"Nature DQN","rank_in_archive_order":36,"of":45,"metrics":{"Score":"429.7"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-battle-zone","task":"Atari Games","dataset":"Atari 2600 Battle Zone","model":"Nature DQN","rank_in_archive_order":29,"of":47,"metrics":{"Score":"26300.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-beam-rider","task":"Atari Games","dataset":"Atari 2600 Beam Rider","model":"Nature DQN","rank_in_archive_order":36,"of":49,"metrics":{"Score":"6846.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-bowling","task":"Atari Games","dataset":"Atari 2600 Bowling","model":"Nature DQN","rank_in_archive_order":36,"of":44,"metrics":{"Score":"42.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-boxing","task":"Atari Games","dataset":"Atari 2600 Boxing","model":"Nature DQN","rank_in_archive_order":36,"of":45,"metrics":{"Score":"71.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-breakout","task":"Atari Games","dataset":"Atari 2600 Breakout","model":"Nature DQN","rank_in_archive_order":27,"of":58,"metrics":{"Score":"401.2"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-centipede","task":"Atari Games","dataset":"Atari 2600 Centipede","model":"Nature DQN","rank_in_archive_order":21,"of":45,"metrics":{"Score":"8309.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-chopper-command","task":"Atari Games","dataset":"Atari 2600 Chopper Command","model":"Nature DQN","rank_in_archive_order":25,"of":45,"metrics":{"Score":"6687.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-crazy-climber","task":"Atari Games","dataset":"Atari 2600 Crazy Climber","model":"Nature DQN","rank_in_archive_order":33,"of":49,"metrics":{"Score":"114103.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-demon-attack","task":"Atari Games","dataset":"Atari 2600 Demon Attack","model":"Nature DQN","rank_in_archive_order":40,"of":46,"metrics":{"Score":"9711.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-double-dunk","task":"Atari Games","dataset":"Atari 2600 Double Dunk","model":"Nature DQN","rank_in_archive_order":27,"of":43,"metrics":{"Score":"-18.1"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-enduro","task":"Atari Games","dataset":"Atari 2600 Enduro","model":"Nature DQN","rank_in_archive_order":37,"of":48,"metrics":{"Score":"301.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-fishing-derby","task":"Atari Games","dataset":"Atari 2600 Fishing Derby","model":"Nature DQN","rank_in_archive_order":37,"of":44,"metrics":{"Score":"-0.8"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-freeway","task":"Atari Games","dataset":"Atari 2600 Freeway","model":"Nature DQN","rank_in_archive_order":33,"of":59,"metrics":{"Score":"30.3"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-frostbite","task":"Atari Games","dataset":"Atari 2600 Frostbite","model":"Nature DQN","rank_in_archive_order":42,"of":53,"metrics":{"Score":"328.3"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gopher","task":"Atari Games","dataset":"Atari 2600 Gopher","model":"Nature DQN","rank_in_archive_order":33,"of":43,"metrics":{"Score":"8520.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-gravitar","task":"Atari Games","dataset":"Atari 2600 Gravitar","model":"Nature DQN","rank_in_archive_order":42,"of":53,"metrics":{"Score":"306.7"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-hero","task":"Atari Games","dataset":"Atari 2600 HERO","model":"Nature DQN","rank_in_archive_order":31,"of":44,"metrics":{"Score":"19950"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-ice-hockey","task":"Atari Games","dataset":"Atari 2600 Ice Hockey","model":"Nature DQN","rank_in_archive_order":21,"of":43,"metrics":{"Score":"-1.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-james-bond","task":"Atari Games","dataset":"Atari 2600 James Bond","model":"Nature DQN","rank_in_archive_order":33,"of":45,"metrics":{"Score":"576.7"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kangaroo","task":"Atari Games","dataset":"Atari 2600 Kangaroo","model":"Nature DQN","rank_in_archive_order":27,"of":47,"metrics":{"Score":"6740.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-krull","task":"Atari Games","dataset":"Atari 2600 Krull","model":"Nature DQN","rank_in_archive_order":42,"of":45,"metrics":{"Score":"3805.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-kung-fu-master","task":"Atari Games","dataset":"Atari 2600 Kung-Fu Master","model":"Nature DQN","rank_in_archive_order":38,"of":43,"metrics":{"Score":"23270.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-montezumas-revenge","task":"Atari Games","dataset":"Atari 2600 Montezuma's Revenge","model":"Nature DQN","rank_in_archive_order":50,"of":50,"metrics":{"Score":"0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-ms-pacman","task":"Atari Games","dataset":"Atari 2600 Ms. Pacman","model":"Nature DQN","rank_in_archive_order":33,"of":47,"metrics":{"Score":"2311.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-name-this-game","task":"Atari Games","dataset":"Atari 2600 Name This Game","model":"Nature DQN","rank_in_archive_order":35,"of":43,"metrics":{"Score":"7257.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-pong","task":"Atari Games","dataset":"Atari 2600 Pong","model":"Nature DQN","rank_in_archive_order":36,"of":52,"metrics":{"Score":"18.9"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-private-eye","task":"Atari Games","dataset":"Atari 2600 Private Eye","model":"Nature DQN","rank_in_archive_order":21,"of":52,"metrics":{"Score":"1788.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-qbert","task":"Atari Games","dataset":"Atari 2600 Q*Bert","model":"Nature DQN","rank_in_archive_order":39,"of":57,"metrics":{"Score":"10596"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-river-raid","task":"Atari Games","dataset":"Atari 2600 River Raid","model":"Nature DQN","rank_in_archive_order":30,"of":41,"metrics":{"Score":"8316.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-road-runner","task":"Atari Games","dataset":"Atari 2600 Road Runner","model":"Nature DQN","rank_in_archive_order":37,"of":44,"metrics":{"Score":"18257.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-robotank","task":"Atari Games","dataset":"Atari 2600 Robotank","model":"Nature DQN","rank_in_archive_order":29,"of":42,"metrics":{"Score":"51.6"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-seaquest","task":"Atari Games","dataset":"Atari 2600 Seaquest","model":"Nature DQN","rank_in_archive_order":32,"of":57,"metrics":{"Score":"5286.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-space-invaders","task":"Atari Games","dataset":"Atari 2600 Space Invaders","model":"Nature DQN","rank_in_archive_order":38,"of":55,"metrics":{"Score":"1976.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-star-gunner","task":"Atari Games","dataset":"Atari 2600 Star Gunner","model":"Nature DQN","rank_in_archive_order":29,"of":43,"metrics":{"Score":"57997.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-star-gunner","task":"Atari Games","dataset":"Atari 2600 Star Gunner","model":"SARSA","rank_in_archive_order":43,"of":43,"metrics":{"Score":"9.4"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-tennis","task":"Atari Games","dataset":"Atari 2600 Tennis","model":"Nature DQN","rank_in_archive_order":29,"of":43,"metrics":{"Score":"-2.5"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-time-pilot","task":"Atari Games","dataset":"Atari 2600 Time Pilot","model":"Nature DQN","rank_in_archive_order":34,"of":44,"metrics":{"Score":"5947.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-tutankham","task":"Atari Games","dataset":"Atari 2600 Tutankham","model":"Nature DQN","rank_in_archive_order":26,"of":44,"metrics":{"Score":"186.7"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-up-and-down","task":"Atari Games","dataset":"Atari 2600 Up and Down","model":"Nature DQN","rank_in_archive_order":39,"of":44,"metrics":{"Score":"8456.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-venture","task":"Atari Games","dataset":"Atari 2600 Venture","model":"Nature DQN","rank_in_archive_order":25,"of":55,"metrics":{"Score":"380.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-video-pinball","task":"Atari Games","dataset":"Atari 2600 Video Pinball","model":"Nature DQN","rank_in_archive_order":36,"of":42,"metrics":{"Score":"42684.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-wizard-of-wor","task":"Atari Games","dataset":"Atari 2600 Wizard of Wor","model":"Nature DQN","rank_in_archive_order":36,"of":41,"metrics":{"Score":"3393.0"},"uses_additional_data":false},{"leaderboard":"/sota/atari-games-on-atari-2600-zaxxon","task":"Atari Games","dataset":"Atari 2600 Zaxxon","model":"Nature DQN","rank_in_archive_order":36,"of":41,"metrics":{"Score":"4977.0"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}