{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/voxelformer-bird-s-eye-view-feature","title":"VoxelFormer: Bird's-Eye-View Feature Generation based on Dual-view Attention for Multi-view 3D Object Detection","arxiv_id":"2304.01054","date":"2023-04-03","proceeding":null,"authors":["Zhuoling Li","Chuanrui Zhang","Wei-Chiu Ma","Yipin Zhou","Linyan Huang","Haoqian Wang","SerNam Lim","Hengshuang Zhao"],"abstract":"In recent years, transformer-based detectors have demonstrated remarkable performance in 2D visual perception tasks. However, their performance in multi-view 3D object detection remains inferior to the state-of-the-art (SOTA) of convolutional neural network based detectors. In this work, we investigate this issue from the perspective of bird's-eye-view (BEV) feature generation. Specifically, we examine the BEV feature generation method employed by the transformer-based SOTA, BEVFormer, and identify its two limitations: (i) it only generates attention weights from BEV, which precludes the use of lidar points for supervision, and (ii) it aggregates camera view features to the BEV through deformable sampling, which only selects a small subset of features and fails to exploit all information. To overcome these limitations, we propose a novel BEV feature generation method, dual-view attention, which generates attention weights from both the BEV and camera view. This method encodes all camera features into the BEV feature. By combining dual-view attention with the BEVFormer architecture, we build a new detector named VoxelFormer. Extensive experiments are conducted on the nuScenes benchmark to verify the superiority of dual-view attention and VoxelForer. We observe that even only adopting 3 encoders and 1 historical frame during training, VoxelFormer still outperforms BEVFormer significantly. When trained in the same setting, VoxelFormer can surpass BEVFormer by 4.9% NDS point. Code is available at: https://github.com/Lizhuoling/VoxelFormer-public.git.","url_abs":"https://arxiv.org/abs/2304.01054v1","url_pdf":"https://arxiv.org/pdf/2304.01054v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"voxelformer-bird-s-eye-view-feature","repo_url":"https://github.com/lizhuoling/voxelformer-public","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2304.01054","atlas_url":"https://app.syntology.ai/?focus=2304.01054","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.01054"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lizhuoling/voxelformer-public","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c637c42a302bb63c","entry":"conv1x1","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/models/backbones/vovnet.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/models/backbones/vovnet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c637c42a302bb63c"}},{"code_sha256_prefix":"303f4d3695ad18f1","entry":"conv3x3","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/models/backbones/vovnet.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/models/backbones/vovnet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"303f4d3695ad18f1"}},{"code_sha256_prefix":"14ddbb72bd9023bf","entry":"denormalize_bbox","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/core/bbox/util.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/core/bbox/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14ddbb72bd9023bf"}},{"code_sha256_prefix":"e6ab81c2ff932920","entry":"normalize_bbox","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/core/bbox/util.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/core/bbox/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e6ab81c2ff932920"}},{"code_sha256_prefix":"718ca707584a0928","entry":"pos2posemb3d","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/models/dense_heads/voxelformer_head.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/models/dense_heads/voxelformer_head.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"718ca707584a0928"}},{"code_sha256_prefix":"c5537a52f3c7aad0","entry":"array_converter","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/core/bbox/array_converter.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/core/bbox/array_converter.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c5537a52f3c7aad0"}},{"code_sha256_prefix":"5ebf15e6940f6710","entry":"dw_conv3x3","repo":"lizhuoling/voxelformer-public","repo_kind":"official","path":"projects/mmdet3d_plugin/models/backbones/vovnet.py","file_url":"https://github.com/lizhuoling/voxelformer-public/blob/HEAD/projects/mmdet3d_plugin/models/backbones/vovnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5ebf15e6940f6710"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}