{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/biformer-vision-transformer-with-bi-level","title":"BiFormer: Vision Transformer with Bi-Level Routing Attention","arxiv_id":"2303.08810","date":"2023-03-15","proceeding":"CVPR 2023 1","authors":["Lei Zhu","Xinjiang Wang","Zhanghan Ke","Wayne Zhang","Rynson Lau"],"abstract":"As the core building block of vision transformers, attention is a powerful tool to capture long-range dependency. However, such power comes at a cost: it incurs a huge computation burden and heavy memory footprint as pairwise token interaction across all spatial locations is computed. A series of works attempt to alleviate this problem by introducing handcrafted and content-agnostic sparsity into attention, such as restricting the attention operation to be inside local windows, axial stripes, or dilated windows. In contrast to these approaches, we propose a novel dynamic sparse attention via bi-level routing to enable a more flexible allocation of computations with content awareness. Specifically, for a query, irrelevant key-value pairs are first filtered out at a coarse region level, and then fine-grained token-to-token attention is applied in the union of remaining candidate regions (\\ie, routed regions). We provide a simple yet effective implementation of the proposed bi-level routing attention, which utilizes the sparsity to save both computation and memory while involving only GPU-friendly dense matrix multiplications. Built with the proposed bi-level routing attention, a new general vision transformer, named BiFormer, is then presented. As BiFormer attends to a small subset of relevant tokens in a \\textbf{query adaptive} manner without distraction from other irrelevant ones, it enjoys both good performance and high computational efficiency, especially in dense prediction tasks. Empirical results across several computer vision tasks such as image classification, object detection, and semantic segmentation verify the effectiveness of our design. Code is available at \\url{https://github.com/rayleizhu/BiFormer}.","url_abs":"https://arxiv.org/abs/2303.08810v1","url_pdf":"https://arxiv.org/pdf/2303.08810v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"biformer-vision-transformer-with-bi-level","repo_url":"https://github.com/rayleizhu/biformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"biformer-vision-transformer-with-bi-level","repo_url":"https://github.com/chenller/mmseg-extension","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"biformer-vision-transformer-with-bi-level","repo_url":"https://gitlab.com/birder/birder","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"BiFormer-B* (IN1k ptretrain)","rank_in_archive_order":228,"of":1060,"metrics":{"Top 1 Accuracy":"85.4%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"BiFormer-S* (IN1k ptretrain)","rank_in_archive_order":325,"of":1060,"metrics":{"Top 1 Accuracy":"84.3%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"BiFormer-T (IN1k ptretrain)","rank_in_archive_order":644,"of":1060,"metrics":{"Top 1 Accuracy":"81.4%"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-2017","task":"Object Detection","dataset":"COCO 2017","model":"BiFormer-B (IN1k pretrain, MaskRCNN 12ep)","rank_in_archive_order":16,"of":24,"metrics":{"mAP":"48.6"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-2017","task":"Object Detection","dataset":"COCO 2017","model":"BiFormer-S (IN1k pretrain, MaskRCNN 12ep)","rank_in_archive_order":18,"of":24,"metrics":{"mAP":"47.8"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"BiFormer-B (IN1k pretrain, Upernet 160k)","rank_in_archive_order":93,"of":235,"metrics":{"Validation mIoU":"51.7"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"Upernet-BiFormer-S (IN1k pretrain, Upernet 160k)","rank_in_archive_order":110,"of":235,"metrics":{"Validation mIoU":"50.8"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.08810","atlas_url":"https://app.syntology.ai/?focus=2303.08810","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.08810"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rayleizhu/biformer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://gitlab.com/birder/birder","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chenller/mmseg-extension","reach":null}],"summary":{"ran":6},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1},"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bcfe9a38637858b7","entry":"BiLevelRoutingAttention","repo":"rayleizhu/biformer","repo_kind":"official","path":"ops/bra_legacy.py","file_url":"https://github.com/rayleizhu/biformer/blob/HEAD/ops/bra_legacy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcfe9a38637858b7"}},{"code_sha256_prefix":"a90fa3e7967ab905","entry":"BiLevelRoutingAttention","repo":"chenller/mmseg-extension","repo_kind":"listed","path":"mmsegext/models/backbones/biformer.py","file_url":"https://github.com/chenller/mmseg-extension/blob/HEAD/mmsegext/models/backbones/biformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a90fa3e7967ab905"}},{"code_sha256_prefix":"0ce8acbb79bfdc96","entry":"KVGather","repo":"rayleizhu/biformer","repo_kind":"official","path":"ops/bra_legacy.py","file_url":"https://github.com/rayleizhu/biformer/blob/HEAD/ops/bra_legacy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ce8acbb79bfdc96"}},{"code_sha256_prefix":"d577326a684b4052","entry":"QKVLinear","repo":"rayleizhu/biformer","repo_kind":"official","path":"ops/bra_legacy.py","file_url":"https://github.com/rayleizhu/biformer/blob/HEAD/ops/bra_legacy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d577326a684b4052"}},{"code_sha256_prefix":"d272b520ceb75e5d","entry":"TopkRouting","repo":"rayleizhu/biformer","repo_kind":"official","path":"ops/bra_legacy.py","file_url":"https://github.com/rayleizhu/biformer/blob/HEAD/ops/bra_legacy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d272b520ceb75e5d"}},{"code_sha256_prefix":"7f21ceacdd6fb9d6","entry":"TopkRouting","repo":"chenller/mmseg-extension","repo_kind":"listed","path":"mmsegext/models/backbones/biformer.py","file_url":"https://github.com/chenller/mmseg-extension/blob/HEAD/mmsegext/models/backbones/biformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f21ceacdd6fb9d6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}