{"url":"/dataset/dqn-replay-dataset","name":"DQN Replay Dataset","full_name":null,"description_markdown":"The DQN Replay Dataset was collected as follows:\r\nWe first train a [DQN][nature_dqn] agent, on all 60 [Atari 2600 games][ale]\r\nwith [sticky actions][stochastic_ale] enabled for 200 million frames (standard protocol) and save all of the experience tuples\r\nof *(observation, action, reward, next observation)* (approximately 50 million)\r\nencountered during training.\r\n\r\nThis logged DQN data can be found in the public [GCP bucket][gcp_bucket]\r\n`gs://atari-replay-datasets` which can be downloaded using [`gsutil`][gsutil].\r\nTo install gsutil, follow the instructions [here][gsutil_install].\r\n\r\nAfter installing gsutil, run the command to copy the entire dataset:\r\n\r\n```\r\ngsutil -m cp -R gs://atari-replay-datasets/dqn\r\n```\r\n\r\nTo run the dataset only for a specific Atari 2600 game (*e.g.*, replace `GAME_NAME`\r\nby `Pong` to download the logged DQN replay datasets for the game of Pong),\r\nrun the command:\r\n\r\n```\r\ngsutil -m cp -R gs://atari-replay-datasets/dqn/[GAME_NAME]\r\n```\r\n\r\nThis data can be generated by running the online agents using\r\n[`batch_rl/baselines/train.py`](https://github.com/google-research/batch_rl/blob/master/batch_rl/baselines/train.py) for 200 million frames\r\n(standard protocol). Note that the dataset consists of approximately 50 million\r\nexperience tuples due to frame skipping (*i.e.*, repeating a selected action for\r\n`k` consecutive frames) of 4. The stickiness parameter is set to 0.25, *i.e.*,\r\nthere is 25% chance at every time step that the environment will execute the\r\nagent's previous action again, instead of the agent's new action.\r\n\r\n[nature_dqn]: https://www.nature.com/articles/nature14236?wm=book_wap_0005\r\n[gsutil_install]: https://cloud.google.com/storage/docs/gsutil_install#install\r\n[gsutil]: https://cloud.google.com/storage/docs/gsutil\r\n[batch_rl]: http://tgabel.de/cms/fileadmin/user_upload/documents/Lange_Gabel_EtAl_RL-Book-12.pdf\r\n[stochastic_ale]: https://arxiv.org/abs/1709.06009\r\n[ale]: https://github.com/mgbellemare/Arcade-Learning-Environment\r\n[gcp_bucket]: https://console.cloud.google.com/storage/browser/atari-replay-datasets\r\n[project_page]: https://offline-rl.github.io","description_withheld":null,"homepage":"https://offline-rl.github.io/","introduced_date":"2019-07-10","introduced_date_note":null,"introduced_by":{"paper":"/paper/striving-for-simplicity-in-off-policy-deep","title":"An Optimistic Perspective on Offline Reinforcement Learning","first_author":"Rishabh Agarwal","url":null},"license":{"name":"CC BY-SA 4.0","url":"https://creativecommons.org/licenses/by-sa/4.0/"},"modalities":[{"name":"Images","url":"/datasets/modality/images"}],"tasks":[{"name":"Atari Games","url":"/task/atari-games","datasets_with_task":"/datasets/task/atari-games"}],"languages":[],"variants":["Atari-57","DQN Replay Dataset"],"data_loaders":[{"repo":"https://github.com/google-research/batch_rl","url":"https://research.google/tools/datasets/dqn-replay/","frameworks":["tf","jax"]},{"repo":"https://github.com/google-research/rlds","url":"https://tensorflow.google.cn/datasets/catalog/rlu_atari","frameworks":["tf"]},{"repo":"https://github.com/mila-iqia/SGI","url":"https://github.com/mila-iqia/SGI","frameworks":["pytorch"]}],"num_papers_in_archive":31,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}