{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/when-prompt-based-incremental-learning-does","title":"When Prompt-based Incremental Learning Does Not Meet Strong Pretraining","arxiv_id":"2308.10445","date":"2023-08-21","proceeding":"ICCV 2023 1","authors":["Yu-Ming Tang","Yi-Xing Peng","Wei-Shi Zheng"],"abstract":"Incremental learning aims to overcome catastrophic forgetting when learning deep networks from sequential tasks. With impressive learning efficiency and performance, prompt-based methods adopt a fixed backbone to sequential tasks by learning task-specific prompts. However, existing prompt-based methods heavily rely on strong pretraining (typically trained on ImageNet-21k), and we find that their models could be trapped if the potential gap between the pretraining task and unknown future tasks is large. In this work, we develop a learnable Adaptive Prompt Generator (APG). The key is to unify the prompt retrieval and prompt learning processes into a learnable prompt generator. Hence, the whole prompting process can be optimized to reduce the negative effects of the gap between tasks effectively. To make our APG avoid learning ineffective knowledge, we maintain a knowledge pool to regularize APG with the feature distribution of each class. Extensive experiments show that our method significantly outperforms advanced methods in exemplar-free incremental learning without (strong) pretraining. Besides, under strong retraining, our method also has comparable performance to existing prompt-based models, showing that our method can still benefit from pretraining. Codes can be found at https://github.com/TOM-tym/APG","url_abs":"https://arxiv.org/abs/2308.10445v1","url_pdf":"https://arxiv.org/pdf/2308.10445v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"when-prompt-based-incremental-learning-does","repo_url":"https://github.com/tom-tym/apg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"exemplar-free","task_name":"Exemplar-Free"},{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2308.10445","atlas_url":"https://app.syntology.ai/?focus=2308.10445","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.10445"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tom-tym/apg","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/TOM-tym/APG","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":12,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":15,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"81ad48a497957921","entry":"accuracy_per_task","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/metrics.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81ad48a497957921"}},{"code_sha256_prefix":"a7bf19b619aa1501","entry":"check_loss","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/utils.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7bf19b619aa1501"}},{"code_sha256_prefix":"d00822b962f6dbd1","entry":"compute_avg_inc_acc","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/results_utils.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/results_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d00822b962f6dbd1"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/backbones/resnet.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/backbones/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/backbones/resnet.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/backbones/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"d17d58f41b4e0b4f","entry":"cord_metric","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/metrics.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d17d58f41b4e0b4f"}},{"code_sha256_prefix":"f7311a4ad1603f3d","entry":"cosine_similarity","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/distance.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/distance.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7311a4ad1603f3d"}},{"code_sha256_prefix":"9f13ed2fb6c8ca34","entry":"get_optimizer","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/factory.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/factory.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f13ed2fb6c8ca34"}},{"code_sha256_prefix":"f1e6c771b035f8df","entry":"get_template_results","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/results_utils.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/results_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1e6c771b035f8df"}},{"code_sha256_prefix":"f100fe9c3a36d8c5","entry":"old_accuracy","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/metrics.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f100fe9c3a36d8c5"}},{"code_sha256_prefix":"34d0cb89ddcd7cdd","entry":"squared_euclidian_distance","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/distance.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/distance.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34d0cb89ddcd7cdd"}},{"code_sha256_prefix":"1f5d558e108778a2","entry":"stable_cosine_distance","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/distance.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/distance.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f5d558e108778a2"}},{"code_sha256_prefix":"78296d162a14a673","entry":"to_onehot","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/utils.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78296d162a14a673"}},{"code_sha256_prefix":"3c002c14b6cacda1","entry":"to_smooth_labeling","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/lib/utils.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/lib/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c002c14b6cacda1"}},{"code_sha256_prefix":"0db0ec3efa6100c1","entry":"resnet18","repo":"TOM-tym/APG","repo_kind":"official","path":"inclearn/backbones/resnet.py","file_url":"https://github.com/TOM-tym/APG/blob/HEAD/inclearn/backbones/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0db0ec3efa6100c1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}