{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/swin3d-a-pretrained-transformer-backbone-for","title":"Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding","arxiv_id":"2304.06906","date":"2023-04-14","proceeding":null,"authors":["Yu-Qi Yang","Yu-Xiao Guo","Jian-Yu Xiong","Yang Liu","Hao Pan","Peng-Shuai Wang","Xin Tong","Baining Guo"],"abstract":"The use of pretrained backbones with fine-tuning has been successful for 2D vision and natural language processing tasks, showing advantages over task-specific networks. In this work, we introduce a pretrained 3D backbone, called {\\SST}, for 3D indoor scene understanding. We design a 3D Swin transformer as our backbone network, which enables efficient self-attention on sparse voxels with linear memory complexity, making the backbone scalable to large models and datasets. We also introduce a generalized contextual relative positional embedding scheme to capture various irregularities of point signals for improved network performance. We pretrained a large {\\SST} model on a synthetic Structured3D dataset, which is an order of magnitude larger than the ScanNet dataset. Our model pretrained on the synthetic dataset not only generalizes well to downstream segmentation and detection on real 3D point datasets, but also outperforms state-of-the-art methods on downstream tasks with +2.3 mIoU and +2.2 mIoU on S3DIS Area5 and 6-fold semantic segmentation, +1.8 mIoU on ScanNet segmentation (val), +1.9 mAP@0.5 on ScanNet detection, and +8.1 mAP@0.5 on S3DIS detection. A series of extensive ablation studies further validate the scalability, generality, and superior performance enabled by our approach. The code and models are available at https://github.com/microsoft/Swin3D .","url_abs":"https://arxiv.org/abs/2304.06906v3","url_pdf":"https://arxiv.org/pdf/2304.06906v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"swin3d-a-pretrained-transformer-backbone-for","repo_url":"https://github.com/microsoft/swin3d","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"swin3d-a-pretrained-transformer-backbone-for","repo_url":"https://github.com/Pointcept/Pointcept","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"stochastic-depth","method_name":"Stochastic Depth"},{"method_slug":"swin-transformer","method_name":"Swin Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-object-detection-on-s3dis","task":"3D Object Detection","dataset":"S3DIS","model":"Swin3D-L+FCAF3D","rank_in_archive_order":3,"of":7,"metrics":{"mAP@0.25":"72.1","mAP@0.5":"54.0"},"uses_additional_data":true},{"leaderboard":"/sota/3d-object-detection-on-scannetv2","task":"3D Object Detection","dataset":"ScanNetV2","model":"Swin3D-L+CAGroup3D","rank_in_archive_order":7,"of":33,"metrics":{"mAP@0.25":"76.4","mAP@0.5":"63.2"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-s3dis","task":"Semantic Segmentation","dataset":"S3DIS","model":"Swin3D-L","rank_in_archive_order":4,"of":54,"metrics":{"Mean IoU":"79.8","Number of params":"N/A","mAcc":"88.0","oAcc":"92.4"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-s3dis-area5","task":"Semantic Segmentation","dataset":"S3DIS Area5","model":"Swin3D-L","rank_in_archive_order":5,"of":61,"metrics":{"Number of params":"N/A","mAcc":"80.5","mIoU":"74.5","oAcc":"92.7"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-scannet","task":"Semantic Segmentation","dataset":"ScanNet","model":"Swin3D-L","rank_in_archive_order":8,"of":45,"metrics":{"test mIoU":"77.9","val mIoU":"77.5"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.06906","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.06906"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/swin3d","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Pointcept/Pointcept","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fda41c833d92c713","entry":"load_state_with_same_shape","repo":"microsoft/swin3d","repo_kind":"official","path":"Swin3D/models/Swin3D.py","file_url":"https://github.com/microsoft/swin3d/blob/HEAD/Swin3D/models/Swin3D.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fda41c833d92c713"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}