{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sgva-clip-semantic-guided-visual-adapting-of","title":"SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification","arxiv_id":"2211.16191","date":"2022-11-28","proceeding":null,"authors":["Fang Peng","Xiaoshan Yang","Linhui Xiao","YaoWei Wang","Changsheng Xu"],"abstract":"Although significant progress has been made in few-shot learning, most of existing few-shot image classification methods require supervised pre-training on a large amount of samples of base classes, which limits their generalization ability in real world application. Recently, large-scale Vision-Language Pre-trained models (VLPs) have been gaining increasing attention in few-shot learning because they can provide a new paradigm for transferable visual representation learning with easily available text on the Web. However, the VLPs may neglect detailed visual information that is difficult to describe by language sentences, but important for learning an effective classifier to distinguish different images. To address the above problem, we propose a new framework, named Semantic-guided Visual Adapting (SgVA), which can effectively extend vision-language pre-trained models to produce discriminative adapted visual features by comprehensively using an implicit knowledge distillation, a vision-specific contrastive loss, and a cross-modal contrastive loss. The implicit knowledge distillation is designed to transfer the fine-grained cross-modal knowledge to guide the updating of the vision adapter. State-of-the-art results on 13 datasets demonstrate that the adapted visual features can well complement the cross-modal features to improve few-shot image classification.","url_abs":"https://arxiv.org/abs/2211.16191v2","url_pdf":"https://arxiv.org/pdf/2211.16191v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sgva-clip-semantic-guided-visual-adapting-of","repo_url":"https://github.com/FannierPeng/SgVA-CLIP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"few-shot-image-classification","task_name":"Few-Shot Image Classification"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/few-shot-image-classification-on-mini-2","task":"Few-Shot Image Classification","dataset":"Mini-Imagenet 5-way (1-shot)","model":"SgVA-CLIP","rank_in_archive_order":1,"of":105,"metrics":{"Accuracy":"97.95"},"uses_additional_data":true},{"leaderboard":"/sota/few-shot-image-classification-on-mini-3","task":"Few-Shot Image Classification","dataset":"Mini-Imagenet 5-way (5-shot)","model":"SgVA-CLIP","rank_in_archive_order":1,"of":95,"metrics":{"Accuracy":"98.72"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2211.16191","atlas_url":"https://app.syntology.ai/?focus=2211.16191","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}