{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vsa-learning-varied-size-window-attention-in","title":"VSA: Learning Varied-Size Window Attention in Vision Transformers","arxiv_id":"2204.08446","date":"2022-04-18","proceeding":null,"authors":["Qiming Zhang","Yufei Xu","Jing Zhang","DaCheng Tao"],"abstract":"Attention within windows has been widely explored in vision transformers to balance the performance, computation complexity, and memory footprint. However, current models adopt a hand-crafted fixed-size window design, which restricts their capacity of modeling long-term dependencies and adapting to objects of different sizes. To address this drawback, we propose \\textbf{V}aried-\\textbf{S}ize Window \\textbf{A}ttention (VSA) to learn adaptive window configurations from data. Specifically, based on the tokens within each default window, VSA employs a window regression module to predict the size and location of the target window, i.e., the attention area where the key and value tokens are sampled. By adopting VSA independently for each attention head, it can model long-term dependencies, capture rich context from diverse windows, and promote information exchange among overlapped windows. VSA is an easy-to-implement module that can replace the window attention in state-of-the-art representative models with minor modifications and negligible extra computational cost while improving their performance by a large margin, e.g., 1.1\\% for Swin-T on ImageNet classification. In addition, the performance gain increases when using larger images for training and test. Experimental results on more downstream tasks, including object detection, instance segmentation, and semantic segmentation, further demonstrate the superiority of VSA over the vanilla window attention in dealing with objects of different sizes. The code will be released https://github.com/ViTAE-Transformer/ViTAE-VSA.","url_abs":"https://arxiv.org/abs/2204.08446v2","url_pdf":"https://arxiv.org/pdf/2204.08446v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vsa-learning-varied-size-window-attention-in","repo_url":"https://github.com/vitae-transformer/vitae-vsa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"vsa-learning-varied-size-window-attention-in","repo_url":"https://github.com/vitae-transformer/qformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2204.08446","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.08446"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vitae-transformer/vitae-vsa","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vitae-transformer/qformer","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ViTAE-Transformer/ViTAE-VSA","reach":{"status":"ok"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"22ebd1057ed2d0c2","entry":"RectifyCoordsGradient","repo":"vitae-transformer/qformer","repo_kind":"listed","path":"models/swin/swin_transformer_vsa.py","file_url":"https://github.com/vitae-transformer/qformer/blob/HEAD/models/swin/swin_transformer_vsa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22ebd1057ed2d0c2"}},{"code_sha256_prefix":"2f2d2dbf82d3f3db","entry":"VSAAttention","repo":"vitae-transformer/qformer","repo_kind":"listed","path":"models/swin/swin_transformer_vsa.py","file_url":"https://github.com/vitae-transformer/qformer/blob/HEAD/models/swin/swin_transformer_vsa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f2d2dbf82d3f3db"}},{"code_sha256_prefix":"e01b15632fdf3e8d","entry":"VSAWindowAttention","repo":"ViTAE-Transformer/ViTAE-VSA","repo_kind":"official","path":"Image-Classification/vitaev2_vsa/vsa.py","file_url":"https://github.com/ViTAE-Transformer/ViTAE-VSA/blob/HEAD/Image-Classification/vitaev2_vsa/vsa.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e01b15632fdf3e8d"}},{"code_sha256_prefix":"477da5846437d6e9","entry":"calc_rel_pos_spatial","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"477da5846437d6e9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}