{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarks-for-deep-off-policy-evaluation-1","title":"Benchmarks for Deep Off-Policy Evaluation","arxiv_id":"2103.16596","date":"2021-03-30","proceeding":"ICLR 2021 1","authors":["Justin Fu","Mohammad Norouzi","Ofir Nachum","George Tucker","Ziyu Wang","Alexander Novikov","Mengjiao Yang","Michael R. Zhang","Yutian Chen","Aviral Kumar","Cosmin Paduraru","Sergey Levine","Tom Le Paine"],"abstract":"Off-policy evaluation (OPE) holds the promise of being able to leverage large, offline datasets for both evaluating and selecting complex policies for decision making. The ability to learn offline is particularly important in many real-world domains, such as in healthcare, recommender systems, or robotics, where online data collection is an expensive and potentially dangerous process. Being able to accurately evaluate and select high-performing policies without requiring online interaction could yield significant benefits in safety, time, and cost for these applications. While many OPE methods have been proposed in recent years, comparing results between papers is difficult because currently there is a lack of a comprehensive and unified benchmark, and measuring algorithmic progress has been challenging due to the lack of difficult evaluation tasks. In order to address this gap, we present a collection of policies that in conjunction with existing offline datasets can be used for benchmarking off-policy evaluation. Our tasks include a range of challenging high-dimensional continuous control problems, with wide selections of datasets and policies for performing policy selection. The goal of our benchmark is to provide a standardized measure of progress that is motivated from a set of principles designed to challenge and test the limits of existing OPE methods. We perform an evaluation of state-of-the-art algorithms and provide open-source access to our data and code to foster future research in this area.","url_abs":"https://arxiv.org/abs/2103.16596v1","url_pdf":"https://arxiv.org/pdf/2103.16596v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarks-for-deep-off-policy-evaluation-1","repo_url":"https://github.com/google-research/deep_ope","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"benchmarks-for-deep-off-policy-evaluation-1","repo_url":"https://github.com/tedmoskovitz/DOPE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"benchmarks-for-deep-off-policy-evaluation-1","repo_url":"https://github.com/tedmoskovitz/TOP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"off-policy-evaluation","task_name":"Off-policy evaluation"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"continuous-control","task_name":"continuous-control"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2103.16596","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.16596"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/google-research/deep_","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/deep_ope","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tedmoskovitz/DOPE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tedmoskovitz/TOP","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"5bb744e06e0b22f8","entry":"calculate_huber_loss","repo":"tedmoskovitz/TOP","repo_kind":"listed","path":"dmc/top_curl_sac.py","file_url":"https://github.com/tedmoskovitz/TOP/blob/HEAD/dmc/top_curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5bb744e06e0b22f8"}},{"code_sha256_prefix":"c01f6250c42168bc","entry":"calculate_quantile_huber_loss","repo":"tedmoskovitz/TOP","repo_kind":"listed","path":"dmc/top_curl_sac.py","file_url":"https://github.com/tedmoskovitz/TOP/blob/HEAD/dmc/top_curl_sac.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c01f6250c42168bc"}},{"code_sha256_prefix":"daa7b3a0355eb7c3","entry":"gaussian_logprob","repo":"tedmoskovitz/DOPE","repo_kind":"listed","path":"dmc/top_curl_sac.py","file_url":"https://github.com/tedmoskovitz/DOPE/blob/HEAD/dmc/top_curl_sac.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"daa7b3a0355eb7c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}