{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-few-shot-adaptation-of-vision","title":"Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages","arxiv_id":"2503.11609","date":"2025-03-14","proceeding":"CVPR 2025 1","authors":["Matteo Farina","Massimiliano Mancini","Giovanni Iacca","Elisa Ricci"],"abstract":"An old-school recipe for training a classifier is to (i) learn a good feature extractor and (ii) optimize a linear layer atop. When only a handful of samples are available per category, as in Few-Shot Adaptation (FSA), data are insufficient to fit a large number of parameters, rendering the above impractical. This is especially true with large pre-trained Vision-Language Models (VLMs), which motivated successful research at the intersection of Parameter-Efficient Fine-tuning (PEFT) and FSA. In this work, we start by analyzing the learning dynamics of PEFT techniques when trained on few-shot data from only a subset of categories, referred to as the ``base'' classes. We show that such dynamics naturally splits into two distinct phases: (i) task-level feature extraction and (ii) specialization to the available concepts. To accommodate this dynamic, we then depart from prompt- or adapter-based methods and tackle FSA differently. Specifically, given a fixed computational budget, we split it to (i) learn a task-specific feature extractor via PEFT and (ii) train a linear classifier on top. We call this scheme Two-Stage Few-Shot Adaptation (2SFS). Differently from established methods, our scheme enables a novel form of selective inference at a category level, i.e., at test time, only novel categories are embedded by the adapted text encoder, while embeddings of base categories are available within the classifier. Results with fixed hyperparameters across two settings, three backbones, and eleven datasets, show that 2SFS matches or surpasses the state-of-the-art, while established methods degrade significantly across settings.","url_abs":"https://arxiv.org/abs/2503.11609v1","url_pdf":"https://arxiv.org/pdf/2503.11609v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-few-shot-adaptation-of-vision","repo_url":"https://github.com/farinamatteo/rethinking_fewshot_vlms","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"linear-layer","method_name":"Linear Layer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.11609","atlas_url":"https://app.syntology.ai/?focus=2503.11609","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.11609"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/farinamatteo/rethinking_fewshot_vlms","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran_honours":1,"unverified":8},"by_repo_kind":{"official":{"samples":12,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ea9a2741a89e9fa3","entry":"named_modules_with_index","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea9a2741a89e9fa3"}},{"code_sha256_prefix":"3a2c8ca13520da36","entry":"num_params","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a2c8ca13520da36"}},{"code_sha256_prefix":"df1779ed67a55033","entry":"trainable_bias_params","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df1779ed67a55033"}},{"code_sha256_prefix":"48286bdf4729b466","entry":"trainable_norm_params","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48286bdf4729b466"}},{"code_sha256_prefix":"05476a4a6b11e351","entry":"SingleStreamClassifier","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05476a4a6b11e351"}},{"code_sha256_prefix":"f2a0a5061e7c204e","entry":"cls_acc","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2a0a5061e7c204e"}},{"code_sha256_prefix":"e71d359de1d35c3e","entry":"evaluate_selective_inference","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e71d359de1d35c3e"}},{"code_sha256_prefix":"9a1a4092f9561d8e","entry":"prepare_for_first_stage","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a1a4092f9561d8e"}},{"code_sha256_prefix":"8291595692fbd0fc","entry":"run_twostage","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8291595692fbd0fc"}},{"code_sha256_prefix":"8541d2e091dc64a9","entry":"tokenize_texts","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8541d2e091dc64a9"}},{"code_sha256_prefix":"6f0b30003b7c4c5c","entry":"train_epoch","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f0b30003b7c4c5c"}},{"code_sha256_prefix":"1d9ea6891b4022ca","entry":"train_epoch_second_stage","repo":"FarinaMatteo/rethinking_fewshot_vlms","repo_kind":"official","path":"fs/twostage.py","file_url":"https://github.com/FarinaMatteo/rethinking_fewshot_vlms/blob/HEAD/fs/twostage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d9ea6891b4022ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}