{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/datasets-for-data-driven-reinforcement","title":"D4RL: Datasets for Deep Data-Driven Reinforcement Learning","arxiv_id":"2004.07219","date":"2020-04-15","proceeding":null,"authors":["Justin Fu","Aviral Kumar","Ofir Nachum","George Tucker","Sergey Levine"],"abstract":"The offline reinforcement learning (RL) setting (also known as full batch RL), where a policy is learned from a static dataset, is compelling as progress enables RL methods to take advantage of large, previously-collected datasets, much like how the rise of large datasets has fueled results in supervised learning. However, existing online RL benchmarks are not tailored towards the offline setting and existing offline RL benchmarks are restricted to data generated by partially-trained agents, making progress in offline RL difficult to measure. In this work, we introduce benchmarks specifically designed for the offline setting, guided by key properties of datasets relevant to real-world applications of offline RL. With a focus on dataset collection, examples of such properties include: datasets generated via hand-designed controllers and human demonstrators, multitask datasets where an agent performs different tasks in the same environment, and datasets collected with mixtures of policies. By moving beyond simple benchmark tasks and data collected by partially-trained RL agents, we reveal important and unappreciated deficiencies of existing algorithms. To facilitate research, we have released our benchmark tasks and datasets with a comprehensive evaluation of existing algorithms, an evaluation protocol, and open-source examples. This serves as a common starting point for the community to identify shortcomings in existing offline RL methods and a collaborative route for progress in this emerging area.","url_abs":"https://arxiv.org/abs/2004.07219v4","url_pdf":"https://arxiv.org/pdf/2004.07219v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/rail-berkeley/d4rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/rail-berkeley/offline_rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/anuragajay/d4rl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/farama-foundation/d4rl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/koulanurag/opcc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/kpertsch/d4rl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"datasets-for-data-driven-reinforcement","repo_url":"https://github.com/mpatacchiola/imujoco","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"d4rl","task_name":"D4RL"},{"task_slug":"offline-rl","task_name":"Offline RL"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[{"slug":"d4rl","name":"D4RL","full_name":"D4RL"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.07219","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.07219"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rail-berkeley/d4rl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mpatacchiola/imujoco","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rail-berkeley/offline_rl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/farama-foundation/d4rl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/koulanurag/opcc","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kpertsch/d4rl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/anuragajay/d4rl","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"unverified":8},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1},"listed":{"samples":11,"ran":4,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ccc25780948874cb","entry":"clamp","repo":"anuragajay/d4rl","repo_kind":"listed","path":"d4rl/carla/carla_env.py","file_url":"https://github.com/anuragajay/d4rl/blob/HEAD/d4rl/carla/carla_env.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ccc25780948874cb"}},{"code_sha256_prefix":"0817d0a04cd3f2f1","entry":"get_returns","repo":"kpertsch/d4rl","repo_kind":"listed","path":"d4rl/ope.py","file_url":"https://github.com/kpertsch/d4rl/blob/HEAD/d4rl/ope.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0817d0a04cd3f2f1"}},{"code_sha256_prefix":"ea95f69594dd2e9c","entry":"parse_maze","repo":"anuragajay/d4rl","repo_kind":"listed","path":"d4rl/pointmaze/maze_model.py","file_url":"https://github.com/anuragajay/d4rl/blob/HEAD/d4rl/pointmaze/maze_model.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ea95f69594dd2e9c"}},{"code_sha256_prefix":"ec64670acc5c32df","entry":"ranking_correlation_metric","repo":"kpertsch/d4rl","repo_kind":"listed","path":"d4rl/ope.py","file_url":"https://github.com/kpertsch/d4rl/blob/HEAD/d4rl/ope.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ec64670acc5c32df"}},{"code_sha256_prefix":"b7eec81dfe2b1b23","entry":"compute_distance","repo":"anuragajay/d4rl","repo_kind":"listed","path":"d4rl/carla/carla_env.py","file_url":"https://github.com/anuragajay/d4rl/blob/HEAD/d4rl/carla/carla_env.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b7eec81dfe2b1b23"}},{"code_sha256_prefix":"b5b4d8429fac0ada","entry":"compute_distance","repo":"anuragajay/d4rl","repo_kind":"listed","path":"d4rl/carla/data_collection_agent_lane.py","file_url":"https://github.com/anuragajay/d4rl/blob/HEAD/d4rl/carla/data_collection_agent_lane.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b5b4d8429fac0ada"}},{"code_sha256_prefix":"88d6d21f699682b1","entry":"download_dataset_from_url","repo":"farama-foundation/d4rl","repo_kind":"listed","path":"d4rl/offline_env.py","file_url":"https://github.com/farama-foundation/d4rl/blob/HEAD/d4rl/offline_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"88d6d21f699682b1"}},{"code_sha256_prefix":"e14321aa0b5c0b78","entry":"filepath_from_url","repo":"farama-foundation/d4rl","repo_kind":"listed","path":"d4rl/offline_env.py","file_url":"https://github.com/farama-foundation/d4rl/blob/HEAD/d4rl/offline_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e14321aa0b5c0b78"}},{"code_sha256_prefix":"395635db622171fa","entry":"get_keys","repo":"rail-berkeley/offline_rl","repo_kind":"official","path":"d4rl/offline_env.py","file_url":"https://github.com/rail-berkeley/offline_rl/blob/HEAD/d4rl/offline_env.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"395635db622171fa"}},{"code_sha256_prefix":"79dface7ed4101c8","entry":"get_turn","repo":"anuragajay/d4rl","repo_kind":"listed","path":"d4rl/gym_minigrid/fourroom_controller.py","file_url":"https://github.com/anuragajay/d4rl/blob/HEAD/d4rl/gym_minigrid/fourroom_controller.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"79dface7ed4101c8"}},{"code_sha256_prefix":"de9c8fb5e38fa8ed","entry":"is_within_distance","repo":"anuragajay/d4rl","repo_kind":"listed","path":"d4rl/carla/carla_env.py","file_url":"https://github.com/anuragajay/d4rl/blob/HEAD/d4rl/carla/carla_env.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de9c8fb5e38fa8ed"}},{"code_sha256_prefix":"3cf6d03023b905d7","entry":"load_policy","repo":"farama-foundation/d4rl","repo_kind":"listed","path":"scripts/generation/generate_ant_maze_datasets.py","file_url":"https://github.com/farama-foundation/d4rl/blob/HEAD/scripts/generation/generate_ant_maze_datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3cf6d03023b905d7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}