{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-tuning-towards-parameter-and","title":"Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation","arxiv_id":"2403.11808","date":"2024-03-18","proceeding":null,"authors":["Wangbo Zhao","Jiasheng Tang","Yizeng Han","Yibing Song","Kai Wang","Gao Huang","Fan Wang","Yang You"],"abstract":"Existing parameter-efficient fine-tuning (PEFT) methods have achieved significant success on vision transformers (ViTs) adaptation by improving parameter efficiency. However, the exploration of enhancing inference efficiency during adaptation remains underexplored. This limits the broader application of pre-trained ViT models, especially when the model is computationally extensive. In this paper, we propose Dynamic Tuning (DyT), a novel approach to improve both parameter and inference efficiency for ViT adaptation. Specifically, besides using the lightweight adapter modules, we propose a token dispatcher to distinguish informative tokens from less important ones, allowing the latter to dynamically skip the original block, thereby reducing the redundant computation during inference. Additionally, we explore multiple design variants to find the best practice of DyT. Finally, inspired by the mixture-of-experts (MoE) mechanism, we introduce an enhanced adapter to further boost the adaptation performance. We validate DyT across various tasks, including image/video recognition and semantic segmentation. For instance, DyT achieves superior performance compared to existing PEFT methods while evoking only 71% of their FLOPs on the VTAB-1K benchmark.","url_abs":"https://arxiv.org/abs/2403.11808v2","url_pdf":"https://arxiv.org/pdf/2403.11808v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-tuning-towards-parameter-and","repo_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-recognition","task_name":"Video Recognition"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.11808","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.11808"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning","reach":{"status":"ok"}}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"3dc19396537db789","entry":"all_reduce_mean","repo":"nus-hpc-ai-lab/dynamic-tuning","repo_kind":"official","path":"misc.py","file_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning/blob/HEAD/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3dc19396537db789"}},{"code_sha256_prefix":"c7eef5c6d9aecac0","entry":"binaray_entropy","repo":"nus-hpc-ai-lab/dynamic-tuning","repo_kind":"official","path":"models/losses.py","file_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning/blob/HEAD/models/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7eef5c6d9aecac0"}},{"code_sha256_prefix":"53d549b4bca5ce6d","entry":"convert_list_to_tensor","repo":"nus-hpc-ai-lab/dynamic-tuning","repo_kind":"official","path":"models/vision_transformer_IN21K.py","file_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning/blob/HEAD/models/vision_transformer_IN21K.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"53d549b4bca5ce6d"}},{"code_sha256_prefix":"39482c6d0eb3ca0a","entry":"select_flops","repo":"nus-hpc-ai-lab/dynamic-tuning","repo_kind":"official","path":"block_flops_dict.py","file_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning/blob/HEAD/block_flops_dict.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39482c6d0eb3ca0a"}},{"code_sha256_prefix":"b0a089f89e71375c","entry":"batch_select_flops","repo":"nus-hpc-ai-lab/dynamic-tuning","repo_kind":"official","path":"block_flops_dict.py","file_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning/blob/HEAD/block_flops_dict.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0a089f89e71375c"}},{"code_sha256_prefix":"1d314401456ed7f5","entry":"get_init_weights_vit","repo":"nus-hpc-ai-lab/dynamic-tuning","repo_kind":"official","path":"models/vision_transformer_IN21K.py","file_url":"https://github.com/nus-hpc-ai-lab/dynamic-tuning/blob/HEAD/models/vision_transformer_IN21K.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1d314401456ed7f5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}