{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adarl-what-where-and-how-to-adapt-in-transfer","title":"AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning","arxiv_id":"2107.02729","date":"2021-07-06","proceeding":"ICLR 2022 4","authors":["Biwei Huang","Fan Feng","Chaochao Lu","Sara Magliacane","Kun Zhang"],"abstract":"One practical challenge in reinforcement learning (RL) is how to make quick adaptations when faced with new environments. In this paper, we propose a principled framework for adaptive RL, called \\textit{AdaRL}, that adapts reliably and efficiently to changes across domains with a few samples from the target domain, even in partially observable environments. Specifically, we leverage a parsimonious graphical representation that characterizes structural relationships over variables in the RL system. Such graphical representations provide a compact way to encode what and where the changes across domains are, and furthermore inform us with a minimal set of changes that one has to consider for the purpose of policy adaptation. We show that by explicitly leveraging this compact representation to encode changes, we can efficiently adapt the policy to the target domain, in which only a few samples are needed and further policy optimization is avoided. We illustrate the efficacy of AdaRL through a series of experiments that vary factors in the observation, transition, and reward functions for Cartpole and Atari games.","url_abs":"https://arxiv.org/abs/2107.02729v4","url_pdf":"https://arxiv.org/pdf/2107.02729v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adarl-what-where-and-how-to-adapt-in-transfer","repo_url":"https://github.com/adaptive-rl/adarl-code","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"atari-games","task_name":"Atari Games"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"transfer-reinforcement-learning","task_name":"Transfer Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2107.02729","atlas_url":"https://app.syntology.ai/?focus=2107.02729","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.02729"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adaptive-rl/adarl-code","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4bcd115491915109","entry":"default_hps","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/misc/hyper_params.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/misc/hyper_params.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4bcd115491915109"}},{"code_sha256_prefix":"ca982b5a7e036ad1","entry":"encode_batch","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/misc/data_handler.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/misc/data_handler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca982b5a7e036ad1"}},{"code_sha256_prefix":"e2eff668b62a975c","entry":"encode_obs","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/policy/extract.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/policy/extract.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2eff668b62a975c"}},{"code_sha256_prefix":"1902625ea14d31a5","entry":"env_init","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/misc/env_init.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/misc/env_init.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1902625ea14d31a5"}},{"code_sha256_prefix":"4ec49be2c9c0cd3d","entry":"extract_theta","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/policy/extract.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/policy/extract.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ec49be2c9c0cd3d"}},{"code_sha256_prefix":"b1ad350eba285138","entry":"load_raw_data_list","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/misc/data_handler.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/misc/data_handler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1ad350eba285138"}},{"code_sha256_prefix":"8071583e658235f1","entry":"next_batch","repo":"adaptive-rl/adarl-code","repo_kind":"official","path":"utils/misc/data_handler.py","file_url":"https://github.com/adaptive-rl/adarl-code/blob/HEAD/utils/misc/data_handler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8071583e658235f1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}