{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/out-of-distribution-dynamics-detection-rl","title":"Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Results","arxiv_id":"2107.04982","date":"2021-07-11","proceeding":null,"authors":["Mohamad H Danesh","Alan Fern"],"abstract":"We study the problem of out-of-distribution dynamics (OODD) detection, which involves detecting when the dynamics of a temporal process change compared to the training-distribution dynamics. This is relevant to applications in control, reinforcement learning (RL), and multi-variate time-series, where changes to test time dynamics can impact the performance of learning controllers/predictors in unknown ways. This problem is particularly important in the context of deep RL, where learned controllers often overfit to the training environment. Currently, however, there is a lack of established OODD benchmarks for the types of environments commonly used in RL research. Our first contribution is to design a set of OODD benchmarks derived from common RL environments with varying types and intensities of OODD. Our second contribution is to design a strong OODD baseline approach based on recurrent implicit quantile network (RIQN), which monitors autoregressive prediction errors for OODD detection. In addition to RIQN, we introduce and test three other simpler baselines. Our final contribution is to evaluate our baseline approaches on the benchmarks to provide results for future comparison.","url_abs":"https://arxiv.org/abs/2107.04982v2","url_pdf":"https://arxiv.org/pdf/2107.04982v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"out-of-distribution-dynamics-detection-rl","repo_url":"https://github.com/modanesh/anomalous_rl_envs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"out-of-distribution-dynamics-detection-rl","repo_url":"https://github.com/modanesh/recurrent_implicit_quantile_networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2107.04982","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.04982"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/modanesh/anomalous_rl_envs","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/modanesh/recurrent_implicit_quantile_networks","reach":null}],"summary":{"ran":1,"ran_fixture":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"a5590ecb0f68e533","entry":"RecurrentIQN","repo":"modanesh/recurrent_implicit_quantile_networks","repo_kind":"official","path":"models.py","file_url":"https://github.com/modanesh/recurrent_implicit_quantile_networks/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5590ecb0f68e533"}},{"code_sha256_prefix":"6b5f1c2ad4ca8a53","entry":"get_action","repo":"modanesh/recurrent_implicit_quantile_networks","repo_kind":"official","path":"autoregressive_control.py","file_url":"https://github.com/modanesh/recurrent_implicit_quantile_networks/blob/HEAD/autoregressive_control.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b5f1c2ad4ca8a53"}},{"code_sha256_prefix":"7a96153b4427a0a4","entry":"test_model","repo":"modanesh/recurrent_implicit_quantile_networks","repo_kind":"official","path":"autoregressive_control.py","file_url":"https://github.com/modanesh/recurrent_implicit_quantile_networks/blob/HEAD/autoregressive_control.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7a96153b4427a0a4"}},{"code_sha256_prefix":"bc1e9012508e7fcd","entry":"train_model","repo":"modanesh/recurrent_implicit_quantile_networks","repo_kind":"official","path":"autoregressive_control.py","file_url":"https://github.com/modanesh/recurrent_implicit_quantile_networks/blob/HEAD/autoregressive_control.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bc1e9012508e7fcd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}