{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mos-model-surgery-for-pre-trained-model-based","title":"MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning","arxiv_id":"2412.09441","date":"2024-12-12","proceeding":null,"authors":["Hai-Long Sun","Da-Wei Zhou","Hanbin Zhao","Le Gan","De-Chuan Zhan","Han-Jia Ye"],"abstract":"Class-Incremental Learning (CIL) requires models to continually acquire knowledge of new classes without forgetting old ones. Despite Pre-trained Models (PTMs) have shown excellent performance in CIL, catastrophic forgetting still occurs as the model learns new concepts. Existing work seeks to utilize lightweight components to adjust the PTM, while the forgetting phenomenon still comes from {\\em parameter and retrieval} levels. Specifically, iterative updates of the model result in parameter drift, while mistakenly retrieving irrelevant modules leads to the mismatch during inference. To this end, we propose MOdel Surgery (MOS) to rescue the model from forgetting previous knowledge. By training task-specific adapters, we continually adjust the PTM to downstream tasks. To mitigate parameter-level forgetting, we present an adapter merging approach to learn task-specific adapters, which aims to bridge the gap between different components while reserve task-specific information. Besides, to address retrieval-level forgetting, we introduce a training-free self-refined adapter retrieval mechanism during inference, which leverages the model's inherent ability for better adapter retrieval. By jointly rectifying the model with those steps, MOS can robustly resist catastrophic forgetting in the learning process. Extensive experiments on seven benchmark datasets validate MOS's state-of-the-art performance. Code is available at: https://github.com/sun-hailong/AAAI25-MOS","url_abs":"https://arxiv.org/abs/2412.09441v1","url_pdf":"https://arxiv.org/pdf/2412.09441v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mos-model-surgery-for-pre-trained-model-based","repo_url":"https://github.com/sun-hailong/aaai25-mos","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"mos-model-surgery-for-pre-trained-model-based","repo_url":"https://github.com/sun-hailong/lamda-pilot","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"class-incremental-learning","task_name":"Class Incremental Learning"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"class-incremental-learning-1","task_name":"class-incremental learning"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.09441","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.09441"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sun-hailong/aaai25-mos","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sun-hailong/lamda-pilot","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_honours":2,"ran_fixture":1,"unverified":11},"by_repo_kind":{"official":{"samples":17,"ran":6,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"sun-hailong/lamda-pilot","repo_kind":"official","path":"backbone/resnet.py","file_url":"https://github.com/sun-hailong/lamda-pilot/blob/HEAD/backbone/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"sun-hailong/lamda-pilot","repo_kind":"official","path":"backbone/resnet.py","file_url":"https://github.com/sun-hailong/lamda-pilot/blob/HEAD/backbone/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"6bdbe3d6368c4cde","entry":"count_parameters","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/toolkit.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/toolkit.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6bdbe3d6368c4cde"}},{"code_sha256_prefix":"4072283c98270784","entry":"reduce_proxies","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"backbone/linears.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/backbone/linears.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4072283c98270784"}},{"code_sha256_prefix":"284e2e0345abe291","entry":"target2onehot","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/toolkit.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/toolkit.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"284e2e0345abe291"}},{"code_sha256_prefix":"929c76a2fa2e8c42","entry":"tensor2numpy","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/toolkit.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/toolkit.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"929c76a2fa2e8c42"}},{"code_sha256_prefix":"cf462ad7948ff43b","entry":"accimage_loader","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/data_manager.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/data_manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cf462ad7948ff43b"}},{"code_sha256_prefix":"b620954495190c1a","entry":"build_transform","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/data.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b620954495190c1a"}},{"code_sha256_prefix":"84339af3a90298e2","entry":"build_transform_coda_prompt","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/data.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"84339af3a90298e2"}},{"code_sha256_prefix":"2c9953e20558c977","entry":"default_loader","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/data_manager.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/data_manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2c9953e20558c977"}},{"code_sha256_prefix":"1228159f65c11926","entry":"get_backbone","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/inc_net.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/inc_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1228159f65c11926"}},{"code_sha256_prefix":"c4068ef3da0e1260","entry":"interpolate_weights","repo":"sun-hailong/lamda-pilot","repo_kind":"official","path":"models/cofima.py","file_url":"https://github.com/sun-hailong/lamda-pilot/blob/HEAD/models/cofima.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4068ef3da0e1260"}},{"code_sha256_prefix":"66b63ed4f327a712","entry":"pil_loader","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"utils/data_manager.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/utils/data_manager.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66b63ed4f327a712"}},{"code_sha256_prefix":"d7c13335c21a1194","entry":"regularized_cn","repo":"sun-hailong/lamda-pilot","repo_kind":"official","path":"models/dgr.py","file_url":"https://github.com/sun-hailong/lamda-pilot/blob/HEAD/models/dgr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7c13335c21a1194"}},{"code_sha256_prefix":"1b21828124579725","entry":"resnet18","repo":"sun-hailong/lamda-pilot","repo_kind":"official","path":"backbone/resnet.py","file_url":"https://github.com/sun-hailong/lamda-pilot/blob/HEAD/backbone/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b21828124579725"}},{"code_sha256_prefix":"b29b6be35c316663","entry":"vit_base_patch16_224_in21k_mos","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"backbone/vit_mos.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/backbone/vit_mos.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b29b6be35c316663"}},{"code_sha256_prefix":"2dc78c999d4d5662","entry":"vit_base_patch16_224_mos","repo":"sun-hailong/aaai25-mos","repo_kind":"official","path":"backbone/vit_mos.py","file_url":"https://github.com/sun-hailong/aaai25-mos/blob/HEAD/backbone/vit_mos.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2dc78c999d4d5662"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}