{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visual-query-tuning-towards-effective-usage","title":"Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning","arxiv_id":"2212.03220","date":"2022-12-06","proceeding":"CVPR 2023 1","authors":["Cheng-Hao Tu","Zheda Mai","Wei-Lun Chao"],"abstract":"Intermediate features of a pre-trained model have been shown informative for making accurate predictions on downstream tasks, even if the model backbone is kept frozen. The key challenge is how to utilize these intermediate features given their gigantic amount. We propose visual query tuning (VQT), a simple yet effective approach to aggregate intermediate features of Vision Transformers. Through introducing a handful of learnable ``query'' tokens to each layer, VQT leverages the inner workings of Transformers to ``summarize'' rich intermediate features of each layer, which can then be used to train the prediction heads of downstream tasks. As VQT keeps the intermediate features intact and only learns to combine them, it enjoys memory efficiency in training, compared to many other parameter-efficient fine-tuning approaches that learn to adapt features and need back-propagation through the entire backbone. This also suggests the complementary role between VQT and those approaches in transfer learning. Empirically, VQT consistently surpasses the state-of-the-art approach that utilizes intermediate features for transfer learning and outperforms full fine-tuning in many cases. Compared to parameter-efficient approaches that adapt features, VQT achieves much higher accuracy under memory constraints. Most importantly, VQT is compatible with these approaches to attain even higher accuracy, making it a simple add-on to further boost transfer learning.","url_abs":"https://arxiv.org/abs/2212.03220v2","url_pdf":"https://arxiv.org/pdf/2212.03220v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-query-tuning-towards-effective-usage","repo_url":"https://github.com/andytu28/VQT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"visual-query-tuning-towards-effective-usage","repo_url":"https://github.com/arkel23/clca","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.03220","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.03220"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/andytu28/VQT","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arkel23/clca","reach":null}],"summary":{"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"900ed88360f3d92f","entry":"resize_pos_embed","repo":"arkel23/clca","repo_kind":"listed","path":"models/vit_vqt.py","file_url":"https://github.com/arkel23/clca/blob/HEAD/models/vit_vqt.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"900ed88360f3d92f"}},{"code_sha256_prefix":"639c2af1a7aac898","entry":"build_h2t_vit_sup_models","repo":"andytu28/VQT","repo_kind":"official","path":"src/models/build_h2t_model.py","file_url":"https://github.com/andytu28/VQT/blob/HEAD/src/models/build_h2t_model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"639c2af1a7aac898"}},{"code_sha256_prefix":"3e009472c0d2fa50","entry":"find_best_lrwd","repo":"andytu28/VQT","repo_kind":"official","path":"tune_vtab.py","file_url":"https://github.com/andytu28/VQT/blob/HEAD/tune_vtab.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"3e009472c0d2fa50"}},{"code_sha256_prefix":"51201372d14e1a56","entry":"find_best_lrwd","repo":"andytu28/VQT","repo_kind":"official","path":"head2toe_sparsity_train.py","file_url":"https://github.com/andytu28/VQT/blob/HEAD/head2toe_sparsity_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"51201372d14e1a56"}},{"code_sha256_prefix":"525deef23c0f7494","entry":"get_lrwd_range","repo":"andytu28/VQT","repo_kind":"official","path":"head2toe_train.py","file_url":"https://github.com/andytu28/VQT/blob/HEAD/head2toe_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"525deef23c0f7494"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}