{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aapl-adding-attributes-to-prompt-learning-for","title":"AAPL: Adding Attributes to Prompt Learning for Vision-Language Models","arxiv_id":"2404.16804","date":"2024-04-25","proceeding":null,"authors":["Gahyeon Kim","Sohee Kim","Seokju Lee"],"abstract":"Recent advances in large pre-trained vision-language models have demonstrated remarkable performance on zero-shot downstream tasks. Building upon this, recent studies, such as CoOp and CoCoOp, have proposed the use of prompt learning, where context within a prompt is replaced with learnable vectors, leading to significant improvements over manually crafted prompts. However, the performance improvement for unseen classes is still marginal, and to tackle this problem, data augmentation has been frequently used in traditional zero-shot learning techniques. Through our experiments, we have identified important issues in CoOp and CoCoOp: the context learned through traditional image augmentation is biased toward seen classes, negatively impacting generalization to unseen classes. To address this problem, we propose adversarial token embedding to disentangle low-level visual augmentation features from high-level class information when inducing bias in learnable prompts. Through our novel mechanism called \"Adding Attributes to Prompt Learning\", AAPL, we guide the learnable context to effectively extract text features by focusing on high-level features for unseen classes. We have conducted experiments across 11 datasets, and overall, AAPL shows favorable performances compared to the existing methods in few-shot learning, zero-shot learning, cross-dataset, and domain generalization tasks.","url_abs":"https://arxiv.org/abs/2404.16804v1","url_pdf":"https://arxiv.org/pdf/2404.16804v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aapl-adding-attributes-to-prompt-learning-for","repo_url":"https://github.com/Gahyeonkim09/AAPL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"image-augmentation","task_name":"Image Augmentation"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[{"method_slug":"coop","method_name":"CoOp"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.16804","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}