{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-interpretability-for-visual-prompt","title":"Exploring Interpretability for Visual Prompt Tuning with Hierarchical Concepts","arxiv_id":"2503.06084","date":"2025-03-08","proceeding":null,"authors":["Yubin Wang","Xinyang Jiang","De Cheng","Xiangqian Zhao","Zilong Wang","Dongsheng Li","Cairong Zhao"],"abstract":"Visual prompt tuning offers significant advantages for adapting pre-trained visual foundation models to specific tasks. However, current research provides limited insight into the interpretability of this approach, which is essential for enhancing AI reliability and enabling AI-driven knowledge discovery. In this paper, rather than learning abstract prompt embeddings, we propose the first framework, named Interpretable Visual Prompt Tuning (IVPT), to explore interpretability for visual prompts, by introducing hierarchical concept prototypes. Specifically, visual prompts are linked to human-understandable semantic concepts, represented as a set of category-agnostic prototypes, each corresponding to a specific region of the image. Then, IVPT aggregates features from these regions to generate interpretable prompts, which are structured hierarchically to explain visual prompts at different granularities. Comprehensive qualitative and quantitative evaluations on fine-grained classification benchmarks show its superior interpretability and performance over conventional visual prompt tuning methods and existing interpretable methods.","url_abs":"https://arxiv.org/abs/2503.06084v1","url_pdf":"https://arxiv.org/pdf/2503.06084v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"visual-prompt-tuning","task_name":"Visual Prompt Tuning"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.06084","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.06084"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ThomasWangY/IVPT","reach":null}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"9f879c7437ead1a8","entry":"AttentionWQKVReturn","repo":"ThomasWangY/IVPT","repo_kind":"found_in_text","path":"models/individual_landmark_vit.py","file_url":"https://github.com/ThomasWangY/IVPT/blob/HEAD/models/individual_landmark_vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f879c7437ead1a8"}},{"code_sha256_prefix":"32c70f43f4e047f7","entry":"IndependentMLPs","repo":"ThomasWangY/IVPT","repo_kind":"found_in_text","path":"models/individual_landmark_vit.py","file_url":"https://github.com/ThomasWangY/IVPT/blob/HEAD/models/individual_landmark_vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32c70f43f4e047f7"}},{"code_sha256_prefix":"45988d6ee6a01afa","entry":"BlockWQKVReturn","repo":"ThomasWangY/IVPT","repo_kind":"found_in_text","path":"models/individual_landmark_vit.py","file_url":"https://github.com/ThomasWangY/IVPT/blob/HEAD/models/individual_landmark_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45988d6ee6a01afa"}},{"code_sha256_prefix":"a79474ab1e7b07f0","entry":"IndividualLandmarkViT","repo":"ThomasWangY/IVPT","repo_kind":"found_in_text","path":"models/individual_landmark_vit.py","file_url":"https://github.com/ThomasWangY/IVPT/blob/HEAD/models/individual_landmark_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a79474ab1e7b07f0"}},{"code_sha256_prefix":"975fed304261dcc2","entry":"compute_attention","repo":"ThomasWangY/IVPT","repo_kind":"found_in_text","path":"models/individual_landmark_vit.py","file_url":"https://github.com/ThomasWangY/IVPT/blob/HEAD/models/individual_landmark_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"975fed304261dcc2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}