{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spatial-temporal-enhanced-transformer-towards","title":"Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection","arxiv_id":"2307.00347","date":"2023-07-01","proceeding":null,"authors":["Yifan Zhang","Zhiyu Zhu","Junhui Hou","Dapeng Wu"],"abstract":"The Detection Transformer (DETR) has revolutionized the design of CNN-based object detection systems, showcasing impressive performance. However, its potential in the domain of multi-frame 3D object detection remains largely unexplored. In this paper, we present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection by addressing three key aspects specifically tailored for this task. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network, which represents queries as nodes in a graph and enables effective modeling of object interactions within a social context. To solve the problem of missing hard cases in the proposed output of the encoder in the current frame, we incorporate the output of the previous frame to initialize the query input of the decoder. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match. And similar queries are insufficiently suppressed and turn into redundant prediction boxes. To address this issue, our proposed IoU regularization term encourages similar queries to be distinct during the refinement. Through extensive experiments, we demonstrate the effectiveness of our approach in handling challenging scenarios, while incurring only a minor additional computational overhead. The code is publicly available at https://github.com/Eaphan/STEMD.","url_abs":"https://arxiv.org/abs/2307.00347v4","url_pdf":"https://arxiv.org/pdf/2307.00347v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spatial-temporal-enhanced-transformer-towards","repo_url":"https://github.com/eaphan/stemd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"graph-attention","task_name":"Graph Attention"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.00347","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.00347"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eaphan/stemd","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":7},"by_repo_kind":{"official":{"samples":8,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e8526a516b4f9206","entry":"convert_basic_c2_names","repo":"eaphan/stemd","repo_kind":"official","path":"efg/utils/d2_model_loading.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/efg/utils/d2_model_loading.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e8526a516b4f9206"}},{"code_sha256_prefix":"4552d09aa9f10a7e","entry":"align_and_update_state_dicts","repo":"eaphan/stemd","repo_kind":"official","path":"efg/utils/d2_model_loading.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/efg/utils/d2_model_loading.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4552d09aa9f10a7e"}},{"code_sha256_prefix":"d65a6f9947376276","entry":"convert_c2_detectron_names","repo":"eaphan/stemd","repo_kind":"official","path":"efg/utils/d2_model_loading.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/efg/utils/d2_model_loading.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d65a6f9947376276"}},{"code_sha256_prefix":"1ee3f1b1247be911","entry":"dn_post_process","repo":"eaphan/stemd","repo_kind":"official","path":"playground/detection.3d/waymo/stemd/STEMD.waymo.resnet18.cdn.epoch12.16f/cdn.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/playground/detection.3d/waymo/stemd/STEMD.waymo.resnet18.cdn.epoch12.16f/cdn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1ee3f1b1247be911"}},{"code_sha256_prefix":"0f46d9d680073a35","entry":"get_kw_to_default_map","repo":"eaphan/stemd","repo_kind":"official","path":"efg/modeling/utils.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/efg/modeling/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f46d9d680073a35"}},{"code_sha256_prefix":"a1e7892e17d509f0","entry":"get_pos_to_kw_map","repo":"eaphan/stemd","repo_kind":"official","path":"efg/modeling/utils.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/efg/modeling/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a1e7892e17d509f0"}},{"code_sha256_prefix":"f1dcdd191e058d41","entry":"get_printer","repo":"eaphan/stemd","repo_kind":"official","path":"efg/modeling/utils.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/efg/modeling/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f1dcdd191e058d41"}},{"code_sha256_prefix":"5e5d396ef5fa54c2","entry":"prepare_for_cdn","repo":"eaphan/stemd","repo_kind":"official","path":"playground/detection.3d/waymo/stemd/STEMD.waymo.resnet18.cdn.epoch12.16f/cdn.py","file_url":"https://github.com/eaphan/stemd/blob/HEAD/playground/detection.3d/waymo/stemd/STEMD.waymo.resnet18.cdn.epoch12.16f/cdn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5e5d396ef5fa54c2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}