{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/consistency-guided-prompt-learning-for-vision","title":"Consistency-guided Prompt Learning for Vision-Language Models","arxiv_id":"2306.01195","date":"2023-06-01","proceeding":null,"authors":["Shuvendu Roy","Ali Etemad"],"abstract":"We propose Consistency-guided Prompt learning (CoPrompt), a new fine-tuning method for vision-language models. Our approach improves the generalization of large foundation models when fine-tuned on downstream tasks in a few-shot setting. The basic idea of CoPrompt is to enforce a consistency constraint in the prediction of the trainable and pre-trained models to prevent overfitting on the downstream task. Additionally, we introduce the following two components into our consistency constraint to further boost the performance: enforcing consistency on two perturbed inputs and combining two dominant paradigms of tuning, prompting and adapter. Enforcing consistency on perturbed input serves to further regularize the consistency constraint, thereby improving generalization. Moreover, the integration of adapters and prompts not only enhances performance on downstream tasks but also offers increased tuning flexibility in both input and output spaces. This facilitates more effective adaptation to downstream tasks in a few-shot learning setting. Experiments show that CoPrompt outperforms existing methods on a range of evaluation suites, including base-to-novel generalization, domain generalization, and cross-dataset evaluation. On generalization, CoPrompt improves the state-of-the-art on zero-shot tasks and the overall harmonic mean over 11 datasets. Detailed ablation studies show the effectiveness of each of the components in CoPrompt. We make our code available at https://github.com/ShuvenduRoy/CoPrompt.","url_abs":"https://arxiv.org/abs/2306.01195v4","url_pdf":"https://arxiv.org/pdf/2306.01195v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"consistency-guided-prompt-learning-for-vision","repo_url":"https://github.com/shuvenduroy/coprompt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"consistency-guided-prompt-learning-for-vision","repo_url":"https://github.com/ShuvenduRoy/FER_TL_PipelineTraining","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"prompt-engineering","task_name":"Prompt Engineering"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/prompt-engineering-on-caltech-101","task":"Prompt Engineering","dataset":"Caltech-101","model":"CoPrompt","rank_in_archive_order":6,"of":14,"metrics":{"Harmonic mean":"96.55"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-dtd","task":"Prompt Engineering","dataset":"DTD","model":"CoPrompt","rank_in_archive_order":5,"of":14,"metrics":{"Harmonic mean":"72.79"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-eurosat","task":"Prompt Engineering","dataset":"EuroSAT","model":"CoPrompt","rank_in_archive_order":5,"of":14,"metrics":{"Harmonic mean":"85.84"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-fgvc-aircraft","task":"Prompt Engineering","dataset":"FGVC-Aircraft","model":"CoPrompt","rank_in_archive_order":9,"of":14,"metrics":{"Harmonic mean":"39.76"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-food-101","task":"Prompt Engineering","dataset":"Food-101","model":"CoPrompt","rank_in_archive_order":2,"of":13,"metrics":{"Harmonic mean":"91.40"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-imagenet","task":"Prompt Engineering","dataset":"ImageNet","model":"CoPrompt","rank_in_archive_order":5,"of":15,"metrics":{"Harmonic mean":"74.33"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-imagenet-a","task":"Prompt Engineering","dataset":"ImageNet-A","model":"CoPrompt","rank_in_archive_order":8,"of":9,"metrics":{"Top-1 accuracy %":"50.50"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-imagenet-r","task":"Prompt Engineering","dataset":"ImageNet-R","model":"CoPrompt","rank_in_archive_order":5,"of":9,"metrics":{"Top-1 accuracy %":"77.51"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-imagenet-s","task":"Prompt Engineering","dataset":"ImageNet-S","model":"CoPrompt","rank_in_archive_order":3,"of":9,"metrics":{"Top-1 accuracy %":"49.43"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-oxford-102-flower","task":"Prompt Engineering","dataset":"Oxford 102 Flower","model":"CoPrompt","rank_in_archive_order":9,"of":14,"metrics":{"Harmonic mean":"85.71"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-oxford-iiit-pet-dataset","task":"Prompt Engineering","dataset":"Oxford-IIIT Pet Dataset","model":"CoPrompt","rank_in_archive_order":3,"of":14,"metrics":{"Harmonic mean":"96.87"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-sun397","task":"Prompt Engineering","dataset":"SUN397","model":"CoPrompt","rank_in_archive_order":2,"of":14,"metrics":{"Harmonic mean":"81.31"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-stanford-cars-1","task":"Prompt Engineering","dataset":"Stanford Cars","model":"CoPrompt","rank_in_archive_order":7,"of":14,"metrics":{"Harmonic mean":"75.66"},"uses_additional_data":false},{"leaderboard":"/sota/prompt-engineering-on-ucf101","task":"Prompt Engineering","dataset":"UCF101","model":"CoPrompt","rank_in_archive_order":7,"of":14,"metrics":{"Harmonic mean":"83.07"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.01195","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.01195"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ShuvenduRoy/FER_TL_PipelineTraining","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shuvenduroy/coprompt","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"clip/simple_tokenizer.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"c47aa9e5b049a11d","entry":"build_model","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"clip/model_clip_original.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/clip/model_clip_original.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c47aa9e5b049a11d"}},{"code_sha256_prefix":"918dfac560aed1c9","entry":"build_model","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"clip/model.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/clip/model.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"918dfac560aed1c9"}},{"code_sha256_prefix":"b45d4ac2bba7e906","entry":"cosine_loss","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"trainers/coprompt.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/trainers/coprompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b45d4ac2bba7e906"}},{"code_sha256_prefix":"fbf8c0143d9c48e3","entry":"load","repo":"shuvenduroy/coprompt","repo_kind":"official","path":"clip/clip.py","file_url":"https://github.com/shuvenduroy/coprompt/blob/HEAD/clip/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbf8c0143d9c48e3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}