{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/continual-task-learning-through-adaptive","title":"Continual Task Learning through Adaptive Policy Self-Composition","arxiv_id":"2411.11364","date":"2024-11-18","proceeding":null,"authors":["Shengchao Hu","YuHang Zhou","Ziqing Fan","Jifeng Hu","Li Shen","Ya zhang","DaCheng Tao"],"abstract":"Training a generalizable agent to continually learn a sequence of tasks from offline trajectories is a natural requirement for long-lived agents, yet remains a significant challenge for current offline reinforcement learning (RL) algorithms. Specifically, an agent must be able to rapidly adapt to new tasks using newly collected trajectories (plasticity), while retaining knowledge from previously learned tasks (stability). However, systematic analyses of this setting are scarce, and it remains unclear whether conventional continual learning (CL) methods are effective in continual offline RL (CORL) scenarios. In this study, we develop the Offline Continual World benchmark and demonstrate that traditional CL methods struggle with catastrophic forgetting, primarily due to the unique distribution shifts inherent to CORL scenarios. To address this challenge, we introduce CompoFormer, a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Upon encountering a new task, CompoFormer leverages semantic correlations to selectively integrate relevant prior policies alongside newly trained parameters, thereby enhancing knowledge sharing and accelerating the learning process. Our experiments reveal that CompoFormer outperforms conventional CL methods, particularly in longer task sequences, showcasing a promising balance between plasticity and stability.","url_abs":"https://arxiv.org/abs/2411.11364v1","url_pdf":"https://arxiv.org/pdf/2411.11364v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"continual-task-learning-through-adaptive","repo_url":"https://github.com/charleshsc/CompoFormer","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"offline-rl","task_name":"Offline RL"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.11364","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.11364"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/charleshsc/CompoFormer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"ran":1,"unverified":9},"by_repo_kind":{"official":{"samples":12,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c00885ffa5fe61e7","entry":"init","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/decision_transformer_grow.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/decision_transformer_grow.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c00885ffa5fe61e7"}},{"code_sha256_prefix":"748385c8184d4945","entry":"init_","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/decision_transformer_grow.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/decision_transformer_grow.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"748385c8184d4945"}},{"code_sha256_prefix":"5cb0c55f58279f59","entry":"simple_separated_format","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"tabulate.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/tabulate.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5cb0c55f58279f59"}},{"code_sha256_prefix":"e188fc03df97419f","entry":"calculate_forgetting","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/Metric.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/Metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e188fc03df97419f"}},{"code_sha256_prefix":"92aa145c46cc37fd","entry":"calculate_mean_performance","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/Metric.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/Metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92aa145c46cc37fd"}},{"code_sha256_prefix":"5e687d63bb9beb4a","entry":"create_exp_name","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"logger.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/logger.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e687d63bb9beb4a"}},{"code_sha256_prefix":"41279dfcda8b39e4","entry":"dict_to_safe_json","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"logger.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/logger.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41279dfcda8b39e4"}},{"code_sha256_prefix":"2be8b4822a8978c6","entry":"is_two_level_nested_list","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/Metric.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/Metric.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2be8b4822a8978c6"}},{"code_sha256_prefix":"00a33466c69c5705","entry":"load_tf_weights_in_gpt2","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/trajectory_gpt2.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/trajectory_gpt2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00a33466c69c5705"}},{"code_sha256_prefix":"5025e676615b5431","entry":"prompt_evaluate_episode","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/prompt_evaluate_episodes.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/prompt_evaluate_episodes.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5025e676615b5431"}},{"code_sha256_prefix":"b0b22be94dde9f35","entry":"prompt_evaluate_episode_rtg","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"dt/prompt_evaluate_episodes.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/dt/prompt_evaluate_episodes.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0b22be94dde9f35"}},{"code_sha256_prefix":"07afff96928bd142","entry":"safe_json","repo":"charleshsc/CompoFormer","repo_kind":"official","path":"logger.py","file_url":"https://github.com/charleshsc/CompoFormer/blob/HEAD/logger.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07afff96928bd142"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}