{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adavit-adaptive-tokens-for-efficient-vision","title":"AdaViT: Adaptive Tokens for Efficient Vision Transformer","arxiv_id":"2112.07658","date":"2021-12-14","proceeding":"CVPR 2022 1","authors":["Hongxu Yin","Arash Vahdat","Jose Alvarez","Arun Mallya","Jan Kautz","Pavlo Molchanov"],"abstract":"We introduce A-ViT, a method that adaptively adjusts the inference cost of vision transformer (ViT) for images of different complexity. A-ViT achieves this by automatically reducing the number of tokens in vision transformers that are processed in the network as inference proceeds. We reformulate Adaptive Computation Time (ACT) for this task, extending halting to discard redundant spatial tokens. The appealing architectural properties of vision transformers enables our adaptive token reduction mechanism to speed up inference without modifying the network architecture or inference hardware. We demonstrate that A-ViT requires no extra parameters or sub-network for halting, as we base the learning of adaptive halting on the original network parameters. We further introduce distributional prior regularization that stabilizes training compared to prior ACT approaches. On the image classification task (ImageNet1K), we show that our proposed A-ViT yields high efficacy in filtering informative spatial features and cutting down on the overall compute. The proposed method improves the throughput of DeiT-Tiny by 62% and DeiT-Small by 38% with only 0.3% accuracy drop, outperforming prior art by a large margin. Project page at https://a-vit.github.io/","url_abs":"https://arxiv.org/abs/2112.07658v3","url_pdf":"https://arxiv.org/pdf/2112.07658v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adavit-adaptive-tokens-for-efficient-vision","repo_url":"https://github.com/NVlabs/A-ViT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"efficient-vits","task_name":"Efficient ViTs"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"token-reduction","task_name":"Token Reduction"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"base","method_name":"BASE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/efficient-vits-on-imagenet-1k-with-deit-s","task":"Efficient ViTs","dataset":"ImageNet-1K (with DeiT-S)","model":"A-ViT","rank_in_archive_order":36,"of":41,"metrics":{"GFLOPs":"3.6","Top 1 Accuracy":"78.6"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2112.07658","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.07658"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NVlabs/A-ViT","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":7},"by_repo_kind":{"listed":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"9153241c4331eb1a","entry":"build_dataset","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9153241c4331eb1a"}},{"code_sha256_prefix":"3bc74137ab36aa79","entry":"build_transform","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"datasets.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3bc74137ab36aa79"}},{"code_sha256_prefix":"5dac11fdc41c896b","entry":"checkpoint_filter_fn","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"timm/models/act_vision_transformer.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/timm/models/act_vision_transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"5dac11fdc41c896b"}},{"code_sha256_prefix":"8ddac231bdda7f9a","entry":"get_distribution_target","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8ddac231bdda7f9a"}},{"code_sha256_prefix":"ff1f438bcfd75b97","entry":"h_to_p","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ff1f438bcfd75b97"}},{"code_sha256_prefix":"7108baaddd2eb47e","entry":"hconcat_resize_min","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"engine_act.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/engine_act.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"7108baaddd2eb47e"}},{"code_sha256_prefix":"1dbef2d4da4ee10f","entry":"resize_pos_embed","repo":"NVlabs/A-ViT","repo_kind":"listed","path":"timm/models/act_vision_transformer.py","file_url":"https://github.com/NVlabs/A-ViT/blob/HEAD/timm/models/act_vision_transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1dbef2d4da4ee10f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}