{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamics-generalisation-in-reinforcement-1","title":"Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policies","arxiv_id":"2310.16686","date":"2023-10-25","proceeding":"NeurIPS 2023 11","authors":["Michael Beukman","Devon Jarvis","Richard Klein","Steven James","Benjamin Rosman"],"abstract":"While reinforcement learning has achieved remarkable successes in several domains, its real-world application is limited due to many methods failing to generalise to unfamiliar conditions. In this work, we consider the problem of generalising to new transition dynamics, corresponding to cases in which the environment's response to the agent's actions differs. For example, the gravitational force exerted on a robot depends on its mass and changes the robot's mobility. Consequently, in such cases, it is necessary to condition an agent's actions on extrinsic state information and pertinent contextual information reflecting how the environment responds. While the need for context-sensitive policies has been established, the manner in which context is incorporated architecturally has received less attention. Thus, in this work, we present an investigation into how context information should be incorporated into behaviour learning to improve generalisation. To this end, we introduce a neural network architecture, the Decision Adapter, which generates the weights of an adapter module and conditions the behaviour of an agent on the context information. We show that the Decision Adapter is a useful generalisation of a previously proposed architecture and empirically demonstrate that it results in superior generalisation performance compared to previous approaches in several environments. Beyond this, the Decision Adapter is more robust to irrelevant distractor variables than several alternative methods.","url_abs":"https://arxiv.org/abs/2310.16686v1","url_pdf":"https://arxiv.org/pdf/2310.16686v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamics-generalisation-in-reinforcement-1","repo_url":"https://github.com/michael-beukman/decisionadapter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"dynamics-generalisation-in-reinforcement-1","repo_url":"https://github.com/tidiane-camaret/contextual_rl_zero_shot","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.16686","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.16686"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tidiane-camaret/contextual_rl_zero_shot","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michael-beukman/decisionadapter","reach":null}],"summary":{"ran":4,"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"2a89fd9080d2d973","entry":"Decoder","repo":"tidiane-camaret/contextual_rl_zero_shot","repo_kind":"listed","path":"meta_rl/iida/predictor.py","file_url":"https://github.com/tidiane-camaret/contextual_rl_zero_shot/blob/HEAD/meta_rl/iida/predictor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a89fd9080d2d973"}},{"code_sha256_prefix":"0a4def33d56c1123","entry":"FeedForward","repo":"tidiane-camaret/contextual_rl_zero_shot","repo_kind":"listed","path":"meta_rl/iida/predictor.py","file_url":"https://github.com/tidiane-camaret/contextual_rl_zero_shot/blob/HEAD/meta_rl/iida/predictor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a4def33d56c1123"}},{"code_sha256_prefix":"e763f5bdda0b944b","entry":"MultipleEncoder","repo":"tidiane-camaret/contextual_rl_zero_shot","repo_kind":"listed","path":"meta_rl/iida/predictor.py","file_url":"https://github.com/tidiane-camaret/contextual_rl_zero_shot/blob/HEAD/meta_rl/iida/predictor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e763f5bdda0b944b"}},{"code_sha256_prefix":"0c6f6a28d742e40a","entry":"Predictor","repo":"tidiane-camaret/contextual_rl_zero_shot","repo_kind":"listed","path":"meta_rl/iida/predictor.py","file_url":"https://github.com/tidiane-camaret/contextual_rl_zero_shot/blob/HEAD/meta_rl/iida/predictor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c6f6a28d742e40a"}},{"code_sha256_prefix":"b0a8d318fa77180b","entry":"swap_if_none","repo":"michael-beukman/decisionadapter","repo_kind":"official","path":"src/genrlise/common/networks/segmented_adapter.py","file_url":"https://github.com/michael-beukman/decisionadapter/blob/HEAD/src/genrlise/common/networks/segmented_adapter.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0a8d318fa77180b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}