{"url":"/method/visformer","slug":"visformer","name":"Visformer","full_name":"Visformer","full_name_withheld":false,"description_markdown":"**Visformer**, or **Vision-friendly Transformer**, is an architecture that combines [Transformer](https://paperswithcode.com/methods/category/transformers)-based architectural features with those from [convolutional neural network](https://paperswithcode.com/methods/category/convolutional-neural-networks) architectures. Visformer adopts the stage-wise design for higher base performance. But [self-attentions](https://paperswithcode.com/method/multi-head-attention) are only utilized in the last two stages, considering that self-attention in the high-resolution stage is relatively inefficient even when the FLOPs are balanced. Visformer employs [bottleneck blocks](https://paperswithcode.com/method/bottleneck-residual-block) in the first stage and utilizes [group 3 × 3 convolutions](https://paperswithcode.com/method/grouped-convolution) in bottleneck blocks inspired by [ResNeXt](https://paperswithcode.com/method/resnext). It also introduces [BatchNorm](https://paperswithcode.com/method/batch-normalization) to patch embedding modules as in CNNs.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Visformer: The Vision-friendly Transformer","paper":"/paper/visformer-the-vision-friendly-transformer","first_author":"Zhengsu Chen","n_authors":6,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/visformer-the-vision-friendly-transformer"},"source":{"url":"https://arxiv.org/abs/2104.12533v4","title":"Visformer: The Vision-friendly Transformer","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Vision Transformers","url":"/methods/category/vision-transformers","pwc_aliases":["vision-transformer"]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/visformer-the-vision-friendly-transformer","title":"Visformer: The Vision-friendly Transformer","date":"2021-04-26","arxiv_id":"2104.12533","n_code_links":5,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}}],"papers_shown":1,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":1}],"tasks_shown":1,"n_tasks":1,"usage_by_year":[{"year":"2021","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/visformer"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}