{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/continual-sequence-generation-with-adaptive","title":"Continual Sequence Generation with Adaptive Compositional Modules","arxiv_id":"2203.10652","date":"2022-03-20","proceeding":"ACL 2022 5","authors":["Yanzhe Zhang","Xuezhi Wang","Diyi Yang"],"abstract":"Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks. Existing work on continual sequence generation either always reuses existing parameters to learn new tasks, which is vulnerable to catastrophic forgetting on dissimilar tasks, or blindly adds new parameters for every new task, which could prevent knowledge sharing between similar tasks. To get the best of both worlds, in this work, we propose continual sequence generation with adaptive compositional modules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks. We also incorporate pseudo experience replay to facilitate knowledge transfer in those shared modules. Experiment results on various sequences of generation tasks show that our framework can adaptively add modules or reuse modules based on task similarity, outperforming state-of-the-art baselines in terms of both performance and parameter efficiency. We make our code public at https://github.com/GT-SALT/Adaptive-Compositional-Modules.","url_abs":"https://arxiv.org/abs/2203.10652v2","url_pdf":"https://arxiv.org/pdf/2203.10652v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"continual-sequence-generation-with-adaptive","repo_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"continual-sequence-generation-with-adaptive","repo_url":"https://github.com/salt-nlp/adaptive-compositional-modules","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"experience-replay","method_name":"Experience Replay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.10652","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.10652"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salt-nlp/adaptive-compositional-modules","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":14},"by_repo_kind":{"official":{"samples":15,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6b5d363ab4dbf012","entry":"to_python_float","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"loss_scaler.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/loss_scaler.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b5d363ab4dbf012"}},{"code_sha256_prefix":"e7014fd797aa4e75","entry":"BN_convert_float","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"fp16util.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/fp16util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7014fd797aa4e75"}},{"code_sha256_prefix":"f887d5ffcbc6ad3b","entry":"conversion_helper","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"fp16.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/fp16.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f887d5ffcbc6ad3b"}},{"code_sha256_prefix":"6063f8fe87afb580","entry":"convert_network","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"fp16util.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/fp16util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6063f8fe87afb580"}},{"code_sha256_prefix":"b5729b6f3daf1fe2","entry":"convert_pytorch_state_dict_to_flax","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"mytransformers/modeling_flax_pytorch_utils.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/mytransformers/modeling_flax_pytorch_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5729b6f3daf1fe2"}},{"code_sha256_prefix":"5f5d99a0dee8c2b1","entry":"convert_tf_weight_name_to_pt_weight_name","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"mytransformers/modeling_tf_pytorch_utils.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/mytransformers/modeling_tf_pytorch_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f5d99a0dee8c2b1"}},{"code_sha256_prefix":"38034aa1d784464d","entry":"fp16_to_fp32","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"fp16.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/fp16.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38034aa1d784464d"}},{"code_sha256_prefix":"ea22baceb9a219a1","entry":"fp32_to_fp16","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"fp16.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/fp16.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea22baceb9a219a1"}},{"code_sha256_prefix":"613d2566737ae9ec","entry":"get_percentile_min_max","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"mytransformers/models/ibert/quant_modules.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/mytransformers/models/ibert/quant_modules.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"613d2566737ae9ec"}},{"code_sha256_prefix":"0f21a55b40b92463","entry":"lcs","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f21a55b40b92463"}},{"code_sha256_prefix":"989078983378be2e","entry":"lcsstr","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"989078983378be2e"}},{"code_sha256_prefix":"b21b359520c9e1fd","entry":"linear_quantize","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"mytransformers/models/ibert/quant_modules.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/mytransformers/models/ibert/quant_modules.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b21b359520c9e1fd"}},{"code_sha256_prefix":"9919cae853c6180f","entry":"network_to_half","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"fp16util.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/fp16util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9919cae853c6180f"}},{"code_sha256_prefix":"b971ea3b818d09b7","entry":"normalize_text","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b971ea3b818d09b7"}},{"code_sha256_prefix":"fca1d8b44670f871","entry":"symmetric_linear_quantization_params","repo":"GT-SALT/Adaptive-Compositional-Modules","repo_kind":"official","path":"mytransformers/models/ibert/quant_modules.py","file_url":"https://github.com/GT-SALT/Adaptive-Compositional-Modules/blob/HEAD/mytransformers/models/ibert/quant_modules.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fca1d8b44670f871"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}