{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/state-space-model-meets-transformer-a-new-1","title":"State Space Model Meets Transformer: A New Paradigm for 3D Object Detection","arxiv_id":"2503.14493","date":"2025-03-18","proceeding":"International Conference on Learning Representations 2025 1","authors":["Chuxin Wang","Wenfei Yang","Xiang Liu","Tianzhu Zhang"],"abstract":"DETR-based methods, which use multi-layer transformer decoders to refine object queries iteratively, have shown promising performance in 3D indoor object detection. However, the scene point features in the transformer decoder remain fixed, leading to minimal contributions from later decoder layers, thereby limiting performance improvement. Recently, State Space Models (SSM) have shown efficient context modeling ability with linear complexity through iterative interactions between system states and inputs. Inspired by SSMs, we propose a new 3D object DEtection paradigm with an interactive STate space model (DEST). In the interactive SSM, we design a novel state-dependent SSM parameterization method that enables system states to effectively serve as queries in 3D indoor detection tasks. In addition, we introduce four key designs tailored to the characteristics of point cloud and SSM: The serialization and bidirectional scanning strategies enable bidirectional feature interaction among scene points within the SSM. The inter-state attention mechanism models the relationships between state points, while the gated feed-forward network enhances inter-channel correlations. To the best of our knowledge, this is the first method to model queries as system states and scene points as system inputs, which can simultaneously update scene point features and query features with linear complexity. Extensive experiments on two challenging datasets demonstrate the effectiveness of our DEST-based method. Our method improves the GroupFree baseline in terms of AP50 on ScanNet V2 (+5.3) and SUN RGB-D (+3.2) datasets. Based on the VDETR baseline, Our method sets a new SOTA on the ScanNetV2 and SUN RGB-D datasets.","url_abs":"https://arxiv.org/abs/2503.14493v1","url_pdf":"https://arxiv.org/pdf/2503.14493v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"state-space-model-meets-transformer-a-new-1","repo_url":"https://github.com/OpenSpaceAI/DEST3D","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"state-space-models","task_name":"State Space Models"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-object-detection-on-sun-rgbd-val","task":"3D Object Detection","dataset":"SUN-RGBD val","model":"DEST (based on V-DETR) (TTA)","rank_in_archive_order":3,"of":32,"metrics":{"mAP@0.25":"69.2","mAP@0.5":"52.2"},"uses_additional_data":false},{"leaderboard":"/sota/3d-object-detection-on-sun-rgbd-val","task":"3D Object Detection","dataset":"SUN-RGBD val","model":"DEST (based on  GroupFree3D)","rank_in_archive_order":11,"of":32,"metrics":{"mAP@0.25":"65.3","mAP@0.5":"48.4"},"uses_additional_data":false},{"leaderboard":"/sota/3d-object-detection-on-scannetv2","task":"3D Object Detection","dataset":"ScanNetV2","model":"DEST (based on V-DETR) (TTA)","rank_in_archive_order":1,"of":33,"metrics":{"mAP@0.25":"78.8","mAP@0.5":"67.9"},"uses_additional_data":false},{"leaderboard":"/sota/3d-object-detection-on-scannetv2","task":"3D Object Detection","dataset":"ScanNetV2","model":"DEST (based on  GroupFree3D)","rank_in_archive_order":15,"of":33,"metrics":{"mAP@0.25":"71.3","mAP@0.5":"58.1"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.14493","atlas_url":"https://app.syntology.ai/?focus=2503.14493","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.14493"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OpenSpaceAI/DEST3D","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_honours":1,"unverified":4},"by_repo_kind":{"listed":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c442c7a276779ddf","entry":"convert_corners_camera2lidar","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/dest_module.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/dest_module.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c442c7a276779ddf"}},{"code_sha256_prefix":"eddd23cbbb297813","entry":"flip_axis_to_camera","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/ap_helper.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/ap_helper.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eddd23cbbb297813"}},{"code_sha256_prefix":"b04696e835b2e2d2","entry":"flip_axis_to_camera_tensor","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/dest_module.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/dest_module.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b04696e835b2e2d2"}},{"code_sha256_prefix":"bb173311aa95fc27","entry":"get_3d_box_batch_tensor","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/dest_module.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/dest_module.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb173311aa95fc27"}},{"code_sha256_prefix":"f7a500c0f491fa7a","entry":"init_to_zero","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"issm_triton/issm_chunk_scan.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/issm_triton/issm_chunk_scan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f7a500c0f491fa7a"}},{"code_sha256_prefix":"fd73be3c7c18abff","entry":"multi_head_attention_forward","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/multi_head_attention.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/multi_head_attention.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fd73be3c7c18abff"}},{"code_sha256_prefix":"d638b0201b9a9ece","entry":"flip_axis_to_depth","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/ap_helper.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/ap_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d638b0201b9a9ece"}},{"code_sha256_prefix":"019b69251d0aed68","entry":"rms_norm_ref","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"issm_triton/layernorm_gated.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/issm_triton/layernorm_gated.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"019b69251d0aed68"}},{"code_sha256_prefix":"0ad12b1e8408e2f9","entry":"set_bn_momentum_default","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"pointnet2/pytorch_utils.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/pointnet2/pytorch_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ad12b1e8408e2f9"}},{"code_sha256_prefix":"1a034a72c15f9903","entry":"softmax","repo":"OpenSpaceAI/DEST3D","repo_kind":"listed","path":"models/ap_helper.py","file_url":"https://github.com/OpenSpaceAI/DEST3D/blob/HEAD/models/ap_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a034a72c15f9903"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}