{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-recurrent-q-learning-for-partially","title":"Deep Recurrent Q-Learning for Partially Observable MDPs","arxiv_id":"1507.06527","date":"2015-07-23","proceeding":null,"authors":["Matthew Hausknecht","Peter Stone"],"abstract":"Deep Reinforcement Learning has yielded proficient controllers for complex\ntasks. However, these controllers have limited memory and rely on being able to\nperceive the complete game screen at each decision point. To address these\nshortcomings, this article investigates the effects of adding recurrency to a\nDeep Q-Network (DQN) by replacing the first post-convolutional fully-connected\nlayer with a recurrent LSTM. The resulting \\textit{Deep Recurrent Q-Network}\n(DRQN), although capable of seeing only a single frame at each timestep,\nsuccessfully integrates information through time and replicates DQN's\nperformance on standard Atari games and partially observed equivalents\nfeaturing flickering game screens. Additionally, when trained with partial\nobservations and evaluated with incrementally more complete observations,\nDRQN's performance scales as a function of observability. Conversely, when\ntrained with full observations and evaluated with partial observations, DRQN's\nperformance degrades less than DQN's. Thus, given the same length of history,\nrecurrency is a viable alternative to stacking a history of frames in the DQN's\ninput layer and while recurrency confers no systematic advantage when learning\nto play the game, the recurrent net can better adapt at evaluation time if the\nquality of observations changes.","url_abs":"http://arxiv.org/abs/1507.06527v4","url_pdf":"http://arxiv.org/pdf/1507.06527v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-recurrent-q-learning-for-partially","repo_url":"https://github.com/Joearrowsmith/hackcambridge_2020","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"deep-recurrent-q-learning-for-partially","repo_url":"https://github.com/hamishs/JAX-RL","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"deep-recurrent-q-learning-for-partially","repo_url":"https://github.com/keep9oing/DRQN-Pytorch-CartPole-v1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-recurrent-q-learning-for-partially","repo_url":"https://github.com/kevslinger/dtqn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-recurrent-q-learning-for-partially","repo_url":"https://github.com/marload/DeepRL-TensorFlow2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1507.06527","atlas_url":"https://app.syntology.ai/?focus=1507.06527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1507.06527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/keep9oing/DRQN-Pytorch-CartPole-v1","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hamishs/JAX-RL","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Joearrowsmith/hackcambridge_2020","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/marload/DeepRL-TensorFlow2","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kevslinger/dtqn","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"10e837671b0dd6de","entry":"forward","repo":"hamishs/JAX-RL","repo_kind":"listed","path":"examples/ppo_cartpole.py","file_url":"https://github.com/hamishs/JAX-RL/blob/HEAD/examples/ppo_cartpole.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"10e837671b0dd6de"}},{"code_sha256_prefix":"ff5795efd681b64a","entry":"forward","repo":"hamishs/JAX-RL","repo_kind":"listed","path":"examples/ppo_cartpole.py","file_url":"https://github.com/hamishs/JAX-RL/blob/HEAD/examples/ppo_cartpole.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff5795efd681b64a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}