{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/odrl-a-benchmark-for-off-dynamics","title":"ODRL: A Benchmark for Off-Dynamics Reinforcement Learning","arxiv_id":"2410.20750","date":"2024-10-28","proceeding":null,"authors":["Jiafei Lyu","Kang Xu","Jiacheng Xu","Mengbei Yan","Jingwen Yang","Zongzhang Zhang","Chenjia Bai","Zongqing Lu","Xiu Li"],"abstract":"We consider off-dynamics reinforcement learning (RL) where one needs to transfer policies across different domains with dynamics mismatch. Despite the focus on developing dynamics-aware algorithms, this field is hindered due to the lack of a standard benchmark. To bridge this gap, we introduce ODRL, the first benchmark tailored for evaluating off-dynamics RL methods. ODRL contains four experimental settings where the source and target domains can be either online or offline, and provides diverse tasks and a broad spectrum of dynamics shifts, making it a reliable platform to comprehensively evaluate the agent's adaptation ability to the target domain. Furthermore, ODRL includes recent off-dynamics RL algorithms in a unified framework and introduces some extra baselines for different settings, all implemented in a single-file manner. To unpack the true adaptation capability of existing methods, we conduct extensive benchmarking experiments, which show that no method has universal advantages across varied dynamics shifts. We hope this benchmark can serve as a cornerstone for future research endeavors. Our code is publicly available at https://github.com/OffDynamicsRL/off-dynamics-rl.","url_abs":"https://arxiv.org/abs/2410.20750v1","url_pdf":"https://arxiv.org/pdf/2410.20750v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"odrl-a-benchmark-for-off-dynamics","repo_url":"https://github.com/offdynamicsrl/off-dynamics-rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.20750","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.20750"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/OffDynamicsRL/off-dynamics-rl","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/offdynamicsrl/off-dynamics-rl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_honours":2,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1e71094b23b75fbe","entry":"AvgL1Norm","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"algo/offline_online/bc_par.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/algo/offline_online/bc_par.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e71094b23b75fbe"}},{"code_sha256_prefix":"82a8f299419d4d83","entry":"asymmetric_l2_loss","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"algo/offline_offline/dara.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/algo/offline_offline/dara.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82a8f299419d4d83"}},{"code_sha256_prefix":"2a67e91feeab6fb8","entry":"eval_policy","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"train_tune.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/train_tune.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a67e91feeab6fb8"}},{"code_sha256_prefix":"3b5a164602274858","entry":"fanin_init","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"algo/utils.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/algo/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b5a164602274858"}},{"code_sha256_prefix":"9910e2fc297f8665","entry":"identity","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"algo/utils.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/algo/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9910e2fc297f8665"}},{"code_sha256_prefix":"6d75afc8832f7e01","entry":"call_algo","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"algo/call_algo.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/algo/call_algo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d75afc8832f7e01"}},{"code_sha256_prefix":"f3d6489d35bb98f8","entry":"call_tune_algo","repo":"offdynamicsrl/off-dynamics-rl","repo_kind":"official","path":"algo/call_tune_algo.py","file_url":"https://github.com/offdynamicsrl/off-dynamics-rl/blob/HEAD/algo/call_tune_algo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3d6489d35bb98f8"}},{"code_sha256_prefix":"ab6b7813b41516d6","entry":"get_normalized_score","repo":"OffDynamicsRL/off-dynamics-rl","repo_kind":"official","path":"envs/infos.py","file_url":"https://github.com/OffDynamicsRL/off-dynamics-rl/blob/HEAD/envs/infos.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab6b7813b41516d6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}