{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-zero-shot-generalization-of-learned","title":"Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation","arxiv_id":"2407.03056","date":"2024-07-03","proceeding":null,"authors":["Marco Mistretta","Alberto Baldrati","Marco Bertini","Andrew D. Bagdanov"],"abstract":"Vision-Language Models (VLMs) demonstrate remarkable zero-shot generalization to unseen tasks, but fall short of the performance of supervised methods in generalizing to downstream tasks with limited data. Prompt learning is emerging as a parameter-efficient method for adapting VLMs, but state-of-the-art approaches require annotated samples. In this paper we propose a novel approach to prompt learning based on unsupervised knowledge distillation from more powerful models. Our approach, which we call Knowledge Distillation Prompt Learning (KDPL), can be integrated into existing prompt learning techniques and eliminates the need for labeled examples during adaptation. Our experiments on more than ten standard benchmark datasets demonstrate that KDPL is very effective at improving generalization of learned prompts for zero-shot domain generalization, zero-shot cross-dataset generalization, and zero-shot base-to-novel class generalization problems. KDPL requires no ground-truth labels for adaptation, and moreover we show that even in the absence of any knowledge of training class names it can be used to effectively transfer knowledge. The code is publicly available at https://github.com/miccunifi/KDPL.","url_abs":"https://arxiv.org/abs/2407.03056v2","url_pdf":"https://arxiv.org/pdf/2407.03056v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-zero-shot-generalization-of-learned","repo_url":"https://github.com/miccunifi/kdpl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.03056","atlas_url":"https://app.syntology.ai/?focus=2407.03056","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.03056"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/miccunifi/kdpl","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran":3,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/clip/simple_tokenizer.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"ba26afd892405335","entry":"compute_ci95","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/parse_test_res.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/parse_test_res.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba26afd892405335"}},{"code_sha256_prefix":"ff34933f02b58a29","entry":"get_K_max","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/trainers/kdpl_utils.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/trainers/kdpl_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff34933f02b58a29"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/clip/simple_tokenizer.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"e49b1a864e601506","entry":"sampling","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/trainers/kdpl_utils.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/trainers/kdpl_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e49b1a864e601506"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/clip/simple_tokenizer.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/clip/simple_tokenizer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"62be2590c6547063","entry":"build_model","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/clip/model.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/clip/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62be2590c6547063"}},{"code_sha256_prefix":"0695164c9caf0b87","entry":"load","repo":"miccunifi/kdpl","repo_kind":"official","path":"src/clip/clip.py","file_url":"https://github.com/miccunifi/kdpl/blob/HEAD/src/clip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0695164c9caf0b87"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}