{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scalable-visual-transformers-with","title":"Scalable Vision Transformers with Hierarchical Pooling","arxiv_id":"2103.10619","date":"2021-03-19","proceeding":"ICCV 2021 10","authors":["Zizheng Pan","Bohan Zhuang","Jing Liu","Haoyu He","Jianfei Cai"],"abstract":"The recently proposed Visual image Transformers (ViT) with pure attention have achieved promising performance on image recognition tasks, such as image classification. However, the routine of the current ViT model is to maintain a full-length patch sequence during inference, which is redundant and lacks hierarchical representation. To this end, we propose a Hierarchical Visual Transformer (HVT) which progressively pools visual tokens to shrink the sequence length and hence reduces the computational cost, analogous to the feature maps downsampling in Convolutional Neural Networks (CNNs). It brings a great benefit that we can increase the model capacity by scaling dimensions of depth/width/resolution/patch size without introducing extra computational complexity due to the reduced sequence length. Moreover, we empirically find that the average pooled visual tokens contain more discriminative information than the single class token. To demonstrate the improved scalability of our HVT, we conduct extensive experiments on the image classification task. With comparable FLOPs, our HVT outperforms the competitive baselines on ImageNet and CIFAR-100 datasets. Code is available at https://github.com/MonashAI/HVT","url_abs":"https://arxiv.org/abs/2103.10619v2","url_pdf":"https://arxiv.org/pdf/2103.10619v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scalable-visual-transformers-with","repo_url":"https://github.com/MonashAI/HVT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"scalable-visual-transformers-with","repo_url":"https://github.com/BR-IDL/PaddleViT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"paddle","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"efficient-vits","task_name":"Efficient ViTs"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/efficient-vits-on-imagenet-1k-with-deit-s","task":"Efficient ViTs","dataset":"ImageNet-1K (with DeiT-S)","model":"HVT-S-1","rank_in_archive_order":38,"of":41,"metrics":{"GFLOPs":"2.7","Top 1 Accuracy":"78.3"},"uses_additional_data":false},{"leaderboard":"/sota/efficient-vits-on-imagenet-1k-with-deit-t","task":"Efficient ViTs","dataset":"ImageNet-1K (with DeiT-T)","model":"HVT-Ti-1","rank_in_archive_order":22,"of":22,"metrics":{"GFLOPs":"0.6","Top 1 Accuracy":"69.6"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"HVT-S-1","rank_in_archive_order":857,"of":1060,"metrics":{"GFLOPs":"2.4","Number of params":"21.74M","Top 1 Accuracy":"78.00%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"HVT-Ti-1","rank_in_archive_order":1026,"of":1060,"metrics":{"GFLOPs":"0.64","Number of params":"5.74M","Top 1 Accuracy":"69.64%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.10619","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.10619"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BR-IDL/PaddleViT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/MonashAI/HVT","reach":null}],"summary":{"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4f3b1c60f898dabe","entry":"Block","repo":"MonashAI/HVT","repo_kind":"official","path":"models.py","file_url":"https://github.com/MonashAI/HVT/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f3b1c60f898dabe"}},{"code_sha256_prefix":"b0278314ed7371be","entry":"HVT","repo":"MonashAI/HVT","repo_kind":"official","path":"models.py","file_url":"https://github.com/MonashAI/HVT/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0278314ed7371be"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}