{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/virtual-sparse-convolution-for-multimodal-3d","title":"Virtual Sparse Convolution for Multimodal 3D Object Detection","arxiv_id":"2303.02314","date":"2023-03-04","proceeding":"CVPR 2023 1","authors":["Hai Wu","Chenglu Wen","Shaoshuai Shi","Xin Li","Cheng Wang"],"abstract":"Recently, virtual/pseudo-point-based 3D object detection that seamlessly fuses RGB images and LiDAR data by depth completion has gained great attention. However, virtual points generated from an image are very dense, introducing a huge amount of redundant computation during detection. Meanwhile, noises brought by inaccurate depth completion significantly degrade detection precision. This paper proposes a fast yet effective backbone, termed VirConvNet, based on a new operator VirConv (Virtual Sparse Convolution), for virtual-point-based 3D object detection. VirConv consists of two key designs: (1) StVD (Stochastic Voxel Discard) and (2) NRConv (Noise-Resistant Submanifold Convolution). StVD alleviates the computation problem by discarding large amounts of nearby redundant voxels. NRConv tackles the noise problem by encoding voxel features in both 2D image and 3D LiDAR space. By integrating VirConv, we first develop an efficient pipeline VirConv-L based on an early fusion design. Then, we build a high-precision pipeline VirConv-T based on a transformed refinement scheme. Finally, we develop a semi-supervised pipeline VirConv-S based on a pseudo-label framework. On the KITTI car 3D detection test leaderboard, our VirConv-L achieves 85% AP with a fast running speed of 56ms. Our VirConv-T and VirConv-S attains a high-precision of 86.3% and 87.2% AP, and currently rank 2nd and 1st, respectively. The code is available at https://github.com/hailanyi/VirConv.","url_abs":"https://arxiv.org/abs/2303.02314v1","url_pdf":"https://arxiv.org/pdf/2303.02314v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"virtual-sparse-convolution-for-multimodal-3d","repo_url":"https://github.com/hailanyi/virconv","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-multi-object-tracking","task_name":"3D Multi-Object Tracking"},{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"depth-completion","task_name":"Depth Completion"},{"task_slug":"multiple-object-tracking","task_name":"Multiple Object Tracking"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"pseudo-label","task_name":"Pseudo Label"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multiple-object-tracking-on-kitti-test-online","task":"Multiple Object Tracking","dataset":"KITTI Test (Online Methods)","model":"VirConvTrack","rank_in_archive_order":7,"of":34,"metrics":{"HOTA":"79.9","IDSW":"201","MOTA":"89.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.02314","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.02314"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hailanyi/VirConv","reach":null}],"summary":{"ran":1,"ran_honours":1,"ran_draft_wrong":1,"unverified":6},"by_repo_kind":{"official":{"samples":9,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"641256c98796f0a5","entry":"X_TRANS","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"641256c98796f0a5"}},{"code_sha256_prefix":"cfb5d7a9b7b762ff","entry":"index2points","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cfb5d7a9b7b762ff"}},{"code_sha256_prefix":"fb89b701924ed8c6","entry":"replace_feature","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fb89b701924ed8c6"}},{"code_sha256_prefix":"1bcb772a293d826d","entry":"NRConvBlock","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1bcb772a293d826d"}},{"code_sha256_prefix":"2817bb45ba222ef7","entry":"VirConv8x","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2817bb45ba222ef7"}},{"code_sha256_prefix":"3f20dcdc51284524","entry":"index2uv","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3f20dcdc51284524"}},{"code_sha256_prefix":"c9bb9171cb692e4a","entry":"layer_voxel_discard","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c9bb9171cb692e4a"}},{"code_sha256_prefix":"c514e44067d93b93","entry":"post_act_block","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c514e44067d93b93"}},{"code_sha256_prefix":"fcb8815617b22031","entry":"post_act_block2d","repo":"hailanyi/VirConv","repo_kind":"official","path":"pcdet/models/backbones_3d/spconv_backbone.py","file_url":"https://github.com/hailanyi/VirConv/blob/HEAD/pcdet/models/backbones_3d/spconv_backbone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fcb8815617b22031"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}