{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/skill-expansion-and-composition-in-parameter","title":"Skill Expansion and Composition in Parameter Space","arxiv_id":"2502.05932","date":"2025-02-09","proceeding":null,"authors":["Tenglong Liu","Jianxiong Li","Yinan Zheng","Haoyi Niu","Yixing Lan","Xin Xu","Xianyuan Zhan"],"abstract":"Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve in response to new challenges like human beings. However, previous methods suffer from limited training efficiency when expanding new skills and fail to fully leverage prior knowledge to facilitate new task learning. In this paper, we propose Parametric Skill Expansion and Composition (PSEC), a new framework designed to iteratively evolve the agents' capabilities and efficiently address new challenges by maintaining a manageable skill library. This library can progressively integrate skill primitives as plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient finetuning, facilitating efficient and flexible skill expansion. This structure also enables the direct skill compositions in parameter space by merging LoRA modules that encode different skills, leveraging shared information across skills to effectively program new skills. Based on this, we propose a context-aware module to dynamically activate different skills to collaboratively handle new tasks. Empowering diverse applications including multi-objective composition, dynamics shift, and continual policy shift, the results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC exhibits superior capacity to leverage prior knowledge to efficiently tackle new challenges, as well as expand its skill libraries to evolve the capabilities. Project website: https://ltlhuuu.github.io/PSEC/.","url_abs":"https://arxiv.org/abs/2502.05932v2","url_pdf":"https://arxiv.org/pdf/2502.05932v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"skill-expansion-and-composition-in-parameter","repo_url":"https://github.com/ltlhuuu/PSEC","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"d4rl","task_name":"D4RL"}],"methods":[{"method_slug":null,"method_name":"Library"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.05932","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.05932"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ltlhuuu/PSEC","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran_honours":3,"ran_violates":2,"ran":2,"unverified":4},"by_repo_kind":{"official":{"samples":12,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"efbad980c5e8e094","entry":"compute_q","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/agents/psec/psec_finetune.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/agents/psec/psec_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efbad980c5e8e094"}},{"code_sha256_prefix":"87d1bc442d93916d","entry":"cosine_beta_schedule","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/networks/diffusion.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/networks/diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87d1bc442d93916d"}},{"code_sha256_prefix":"ad221b0896e1bcd0","entry":"expectile_loss","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/agents/psec/psec_finetune.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/agents/psec/psec_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad221b0896e1bcd0"}},{"code_sha256_prefix":"1049990078a4b0f2","entry":"get_weight_decay_mask","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/networks/mlp.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/networks/mlp.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1049990078a4b0f2"}},{"code_sha256_prefix":"0cd44ae2231fcb3b","entry":"linear_beta_schedule","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/networks/diffusion.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/networks/diffusion.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0cd44ae2231fcb3b"}},{"code_sha256_prefix":"7fc1d127bf81537f","entry":"mish","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/agents/psec/psec_pretrain.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/agents/psec/psec_pretrain.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fc1d127bf81537f"}},{"code_sha256_prefix":"7e821a429b5ff00e","entry":"safe_expectile_loss","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/agents/psec/psec_finetune.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/agents/psec/psec_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e821a429b5ff00e"}},{"code_sha256_prefix":"539aaf559a17a19a","entry":"vp_beta_schedule","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/networks/diffusion.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/networks/diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"539aaf559a17a19a"}},{"code_sha256_prefix":"27b1370715ab6f72","entry":"evaluate","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/evaluation_dsrl.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/evaluation_dsrl.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27b1370715ab6f72"}},{"code_sha256_prefix":"74b0ae56402911be","entry":"evaluate1","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/evaluation_dsrl.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/evaluation_dsrl.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74b0ae56402911be"}},{"code_sha256_prefix":"c7143d30af7311f2","entry":"evaluate2","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/evaluation_dsrl.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/evaluation_dsrl.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7143d30af7311f2"}},{"code_sha256_prefix":"c021b9fd8bb7677d","entry":"subsample_ensemble","repo":"ltlhuuu/PSEC","repo_kind":"official","path":"jaxrl5/networks/ensemble.py","file_url":"https://github.com/ltlhuuu/PSEC/blob/HEAD/jaxrl5/networks/ensemble.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c021b9fd8bb7677d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}