{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaptive-prompt-unlocking-the-power-of-visual","title":"Adaptive Prompt: Unlocking the Power of Visual Prompt Tuning","arxiv_id":"2501.18936","date":"2025-01-31","proceeding":null,"authors":["Minh Le","Anh Nguyen","Huy Nguyen","Chau Nguyen","Nhat Ho"],"abstract":"Visual Prompt Tuning (VPT) has recently emerged as a powerful method for adapting pre-trained vision models to downstream tasks. By introducing learnable prompt tokens as task-specific instructions, VPT effectively guides pre-trained transformer models with minimal overhead. Despite its empirical success, a comprehensive theoretical understanding of VPT remains an active area of research. Building on recent insights into the connection between mixture of experts and prompt-based approaches, we identify a key limitation in VPT: the restricted functional expressiveness in prompt formulation. To address this limitation, we propose Visual Adaptive Prompt Tuning (VAPT), a new generation of prompts that redefines prompts as adaptive functions of the input. Our theoretical analysis shows that this simple yet intuitive approach achieves optimal sample efficiency. Empirical results on VTAB-1K and FGVC further demonstrate VAPT's effectiveness, with performance gains of 7.34% and 1.04% over fully fine-tuning baselines, respectively. Notably, VAPT also surpasses VPT by a substantial margin while using fewer parameters. These results highlight both the effectiveness and efficiency of our method and pave the way for future research to explore the potential of adaptive prompts.","url_abs":"https://arxiv.org/abs/2501.18936v1","url_pdf":"https://arxiv.org/pdf/2501.18936v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"visual-prompt-tuning","task_name":"Visual Prompt Tuning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2501.18936","atlas_url":"https://app.syntology.ai/?focus=2501.18936","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.18936"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Minhchuyentoancbn/VAPT","reach":null}],"summary":{"ran":7,"unverified":5},"by_repo_kind":{"found_in_text":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"0dd334991598ccbc","entry":"Attention","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0dd334991598ccbc"}},{"code_sha256_prefix":"e51dc2d64fc3c533","entry":"Block","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e51dc2d64fc3c533"}},{"code_sha256_prefix":"154841e65d6c1b50","entry":"ChannelWiseConv2d","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"154841e65d6c1b50"}},{"code_sha256_prefix":"0498c7765ac40058","entry":"Mlp","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0498c7765ac40058"}},{"code_sha256_prefix":"6a3e067070499183","entry":"PreActBottleneck","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6a3e067070499183"}},{"code_sha256_prefix":"60e12c45a4b7f153","entry":"PromptDownSample","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"60e12c45a4b7f153"}},{"code_sha256_prefix":"4a78cd4620c50018","entry":"PromptMLP","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4a78cd4620c50018"}},{"code_sha256_prefix":"47f5376080d21919","entry":"Embeddings","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"47f5376080d21919"}},{"code_sha256_prefix":"e1eae54076bf5404","entry":"Encoder","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e1eae54076bf5404"}},{"code_sha256_prefix":"51b8d6937dece2a5","entry":"PromptedTransformerAdaptive","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"51b8d6937dece2a5"}},{"code_sha256_prefix":"36e99294b13668ca","entry":"ResNetV2","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"36e99294b13668ca"}},{"code_sha256_prefix":"07235216ba68394b","entry":"Transformer","repo":"Minhchuyentoancbn/VAPT","repo_kind":"found_in_text","path":"src/models/vit_prompt/vit.py","file_url":"https://github.com/Minhchuyentoancbn/VAPT/blob/HEAD/src/models/vit_prompt/vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"07235216ba68394b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}