{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fastervit-fast-vision-transformers-with","title":"FasterViT: Fast Vision Transformers with Hierarchical Attention","arxiv_id":"2306.06189","date":"2023-06-09","proceeding":null,"authors":["Ali Hatamizadeh","Greg Heinrich","Hongxu Yin","Andrew Tao","Jose M. Alvarez","Jan Kautz","Pavlo Molchanov"],"abstract":"We design a new family of hybrid CNN-ViT neural networks, named FasterViT, with a focus on high image throughput for computer vision (CV) applications. FasterViT combines the benefits of fast local representation learning in CNNs and global modeling properties in ViT. Our newly introduced Hierarchical Attention (HAT) approach decomposes global self-attention with quadratic complexity into a multi-level attention with reduced computational costs. We benefit from efficient window-based self-attention. Each window has access to dedicated carrier tokens that participate in local and global representation learning. At a high level, global self-attentions enable the efficient cross-window communication at lower costs. FasterViT achieves a SOTA Pareto-front in terms of accuracy and image throughput. We have extensively validated its effectiveness on various CV tasks including classification, object detection and segmentation. We also show that HAT can be used as a plug-and-play module for existing networks and enhance them. We further demonstrate significantly faster and more accurate performance than competitive counterparts for images with high resolution. Code is available at https://github.com/NVlabs/FasterViT.","url_abs":"https://arxiv.org/abs/2306.06189v2","url_pdf":"https://arxiv.org/pdf/2306.06189v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fastervit-fast-vision-transformers-with","repo_url":"https://github.com/NVlabs/FasterViT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"fastervit-fast-vision-transformers-with","repo_url":"https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/fastervit","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-6","rank_in_archive_order":199,"of":1060,"metrics":{"GFLOPs":"142","Number of params":"1360M","Top 1 Accuracy":"85.8%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-5","rank_in_archive_order":217,"of":1060,"metrics":{"GFLOPs":"113","Number of params":"957.5M","Top 1 Accuracy":"85.6%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-4","rank_in_archive_order":235,"of":1060,"metrics":{"GFLOPs":"36.6","Number of params":"424.6M","Top 1 Accuracy":"85.4%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-3","rank_in_archive_order":282,"of":1060,"metrics":{"GFLOPs":"18.2","Number of params":"159.5M","Top 1 Accuracy":"84.9%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-2","rank_in_archive_order":340,"of":1060,"metrics":{"GFLOPs":"8.7","Number of params":"75.9M","Top 1 Accuracy":"84.2%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-1","rank_in_archive_order":459,"of":1060,"metrics":{"GFLOPs":"5.3","Number of params":"53.4M","Top 1 Accuracy":"83.2%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FasterViT-0","rank_in_archive_order":578,"of":1060,"metrics":{"GFLOPs":"3.3","Number of params":"31.4M","Top 1 Accuracy":"82.1%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.06189","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.06189"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NVlabs/FasterViT","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/fastervit","reach":null}],"summary":{"ran":1,"ran_fixture":2,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":9,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"7f3339b6431dadb2","entry":"PosEmbMLPSwinv1D","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7f3339b6431dadb2"}},{"code_sha256_prefix":"461e5cfacd06764d","entry":"ct_dewindow","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"461e5cfacd06764d"}},{"code_sha256_prefix":"7072909ab4fb3b25","entry":"window_partition","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7072909ab4fb3b25"}},{"code_sha256_prefix":"1577313e8d258159","entry":"window_reverse","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1577313e8d258159"}},{"code_sha256_prefix":"f357b45220a1fcf9","entry":"HAT","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f357b45220a1fcf9"}},{"code_sha256_prefix":"128ed4df18554274","entry":"Mlp","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"128ed4df18554274"}},{"code_sha256_prefix":"d6a720427c459fa8","entry":"PosEmbMLPSwinv2D","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d6a720427c459fa8"}},{"code_sha256_prefix":"c8568529946c5652","entry":"WindowAttention","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c8568529946c5652"}},{"code_sha256_prefix":"808ccf2de8e71b9d","entry":"ct_window","repo":"NVlabs/FasterViT","repo_kind":"official","path":"fastervit/models/faster_vit.py","file_url":"https://github.com/NVlabs/FasterViT/blob/HEAD/fastervit/models/faster_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"808ccf2de8e71b9d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}