{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-few-shot-adaptation-of-foundation","title":"Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning","arxiv_id":"2402.15017","date":"2024-02-22","proceeding":null,"authors":["Zhuoyan Xu","Zhenmei Shi","Junyi Wei","Fangzhou Mu","Yin Li","YIngyu Liang"],"abstract":"Foundation models have emerged as a powerful tool for many AI problems. Despite the tremendous success of foundation models, effective adaptation to new tasks, particularly those with limited labels, remains an open question and lacks theoretical understanding. An emerging solution with recent success in vision and NLP involves finetuning a foundation model on a selection of relevant tasks, before its adaptation to a target task with limited labeled samples. In this paper, we study the theoretical justification of this multitask finetuning approach. Our theoretical analysis reveals that with a diverse set of related tasks, this multitask finetuning leads to reduced error in the target task, in comparison to directly adapting the same pretrained model. We quantify the relationship between finetuning tasks and target tasks by diversity and consistency metrics, and further propose a practical task selection algorithm. We substantiate our theoretical claims with extensive empirical evidence. Further, we present results affirming our task selection algorithm adeptly chooses related finetuning tasks, providing advantages to the model performance on target tasks. We believe our study shed new light on the effective adaptation of foundation models to new tasks that lack abundant labels. Our code is available at https://github.com/OliverXUZY/Foudation-Model_Multitask.","url_abs":"https://arxiv.org/abs/2402.15017v1","url_pdf":"https://arxiv.org/pdf/2402.15017v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-few-shot-adaptation-of-foundation","repo_url":"https://github.com/oliverxuzy/foudation-model_multitask","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2402.15017","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.15017"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/OliverXUZY/Foudation-Model_Multitask","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oliverxuzy/foudation-model_multitask","reach":{"status":"ok"}}],"summary":{"ran":2,"ran_draft_wrong":3,"ran_honours":2,"unverified":9},"by_repo_kind":{"official":{"samples":16,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"448391a260518c64","entry":"AverageMeter","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"448391a260518c64"}},{"code_sha256_prefix":"103e5bf76abe5b7d","entry":"Timer","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"103e5bf76abe5b7d"}},{"code_sha256_prefix":"57935c9233ae310a","entry":"count_params","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57935c9233ae310a"}},{"code_sha256_prefix":"05f178cf8f67fb2e","entry":"decay_lr","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"05f178cf8f67fb2e"}},{"code_sha256_prefix":"5c0a845766c67c36","entry":"make","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c0a845766c67c36"}},{"code_sha256_prefix":"df8ef801f1f069b0","entry":"time_str","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"df8ef801f1f069b0"}},{"code_sha256_prefix":"b5536ea9ed2c47b3","entry":"warmup","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b5536ea9ed2c47b3"}},{"code_sha256_prefix":"3a86045f15981b84","entry":"accuracy","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3a86045f15981b84"}},{"code_sha256_prefix":"34e85ca54daccd74","entry":"ensure_path","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34e85ca54daccd74"}},{"code_sha256_prefix":"7d408a58c39fd05b","entry":"log","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d408a58c39fd05b"}},{"code_sha256_prefix":"19ca843d619cc77a","entry":"main","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19ca843d619cc77a"}},{"code_sha256_prefix":"db58e5a4a5aafdab","entry":"make","repo":"OliverXUZY/Foudation-Model_Multitask","repo_kind":"official","path":"datasets/datasets.py","file_url":"https://github.com/OliverXUZY/Foudation-Model_Multitask/blob/HEAD/datasets/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"db58e5a4a5aafdab"}},{"code_sha256_prefix":"94b1635e5243cca8","entry":"make_path","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"94b1635e5243cca8"}},{"code_sha256_prefix":"9687bf7a922f098a","entry":"register","repo":"OliverXUZY/Foudation-Model_Multitask","repo_kind":"official","path":"datasets/datasets.py","file_url":"https://github.com/OliverXUZY/Foudation-Model_Multitask/blob/HEAD/datasets/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9687bf7a922f098a"}},{"code_sha256_prefix":"08a4ae2c29e737a1","entry":"set_bn_eval","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"08a4ae2c29e737a1"}},{"code_sha256_prefix":"f4bfac480be197ea","entry":"set_log_path","repo":"oliverxuzy/foudation-model_multitask","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/oliverxuzy/foudation-model_multitask/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4bfac480be197ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}