{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/octformer-octree-based-transformers-for-3d","title":"OctFormer: Octree-based Transformers for 3D Point Clouds","arxiv_id":"2305.03045","date":"2023-05-04","proceeding":null,"authors":["Peng-Shuai Wang"],"abstract":"We propose octree-based transformers, named OctFormer, for 3D point cloud learning. OctFormer can not only serve as a general and effective backbone for 3D point cloud segmentation and object detection but also have linear complexity and is scalable for large-scale point clouds. The key challenge in applying transformers to point clouds is reducing the quadratic, thus overwhelming, computation complexity of attentions. To combat this issue, several works divide point clouds into non-overlapping windows and constrain attentions in each local window. However, the point number in each window varies greatly, impeding the efficient execution on GPU. Observing that attentions are robust to the shapes of local windows, we propose a novel octree attention, which leverages sorted shuffled keys of octrees to partition point clouds into local windows containing a fixed number of points while permitting shapes of windows to change freely. And we also introduce dilated octree attention to expand the receptive field further. Our octree attention can be implemented in 10 lines of code with open-sourced libraries and runs 17 times faster than other point cloud attentions when the point number exceeds 200k. Built upon the octree attention, OctFormer can be easily scaled up and achieves state-of-the-art performances on a series of 3D segmentation and detection benchmarks, surpassing previous sparse-voxel-based CNNs and point cloud transformers in terms of both efficiency and effectiveness. Notably, on the challenging ScanNet200 dataset, OctFormer outperforms sparse-voxel-based CNNs by 7.3 in mIoU. Our code and trained models are available at https://wang-ps.github.io/octformer.","url_abs":"https://arxiv.org/abs/2305.03045v2","url_pdf":"https://arxiv.org/pdf/2305.03045v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"octformer-octree-based-transformers-for-3d","repo_url":"https://github.com/octree-nn/octformer","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"octformer-octree-based-transformers-for-3d","repo_url":"https://github.com/irmvlab/point-mamba","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"octformer-octree-based-transformers-for-3d","repo_url":"https://github.com/MindSpore-scientific-2/code-3/tree/main/Sub2Full-OCT-Denoising","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"octformer-octree-based-transformers-for-3d","repo_url":"https://github.com/Pointcept/Pointcept","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"3d-semantic-segmentation","task_name":"3D Semantic Segmentation"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"point-cloud-segmentation","task_name":"Point Cloud Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-object-detection-on-sun-rgbd-val","task":"3D Object Detection","dataset":"SUN-RGBD val","model":"OctFormer","rank_in_archive_order":9,"of":32,"metrics":{"mAP@0.25":"66.2","mAP@0.5":"50.6"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-segmentation-on-scannet-1","task":"3D Semantic Segmentation","dataset":"ScanNet++","model":"OctFormer","rank_in_archive_order":6,"of":8,"metrics":{"Top-1 IoU":"0.460","Top-3 IoU":"0.691"},"uses_additional_data":false},{"leaderboard":"/sota/3d-semantic-segmentation-on-scannet200","task":"3D Semantic Segmentation","dataset":"ScanNet200","model":"OctFormer","rank_in_archive_order":10,"of":16,"metrics":{"test mIoU":"32.5","val mIoU":"32.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-scannet","task":"Semantic Segmentation","dataset":"ScanNet","model":"OctFormer","rank_in_archive_order":17,"of":45,"metrics":{"test mIoU":"76.6","val mIoU":"75.7"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.03045","atlas_url":"https://app.syntology.ai/?focus=2305.03045","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.03045"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/octree-nn/octformer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/irmvlab/point-mamba","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Pointcept/Pointcept","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific-2/code-3/tree/main/Sub2Full-OCT-Denoising","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1},"listed":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e88553cf36ba696f","entry":"elastic_distort","repo":"octree-nn/octformer","repo_kind":"official","path":"datasets/scannet.py","file_url":"https://github.com/octree-nn/octformer/blob/HEAD/datasets/scannet.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e88553cf36ba696f"}},{"code_sha256_prefix":"080c898be81e46d0","entry":"extract_example","repo":"MindSpore-scientific-2/code-3","repo_kind":"listed","path":"The_RefinedWeb_Dataset_for_Falcon/eval_factuality_mindspore.py","file_url":"https://github.com/MindSpore-scientific-2/code-3/blob/HEAD/The_RefinedWeb_Dataset_for_Falcon/eval_factuality_mindspore.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"080c898be81e46d0"}},{"code_sha256_prefix":"fc721854c3bf3762","entry":"labels2cat","repo":"MindSpore-scientific-2/code-3","repo_kind":"listed","path":"Three-Dimensional-Lip-Motion-Network-for-Text-Independent-Speaker-Recognition-master/3LMNet_ms.py","file_url":"https://github.com/MindSpore-scientific-2/code-3/blob/HEAD/Three-Dimensional-Lip-Motion-Network-for-Text-Independent-Speaker-Recognition-master/3LMNet_ms.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc721854c3bf3762"}},{"code_sha256_prefix":"962937e4c560ebb3","entry":"labels2onehot","repo":"MindSpore-scientific-2/code-3","repo_kind":"listed","path":"Three-Dimensional-Lip-Motion-Network-for-Text-Independent-Speaker-Recognition-master/3LMNet_ms.py","file_url":"https://github.com/MindSpore-scientific-2/code-3/blob/HEAD/Three-Dimensional-Lip-Motion-Network-for-Text-Independent-Speaker-Recognition-master/3LMNet_ms.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"962937e4c560ebb3"}},{"code_sha256_prefix":"9c1b2df7fb7a5f6f","entry":"onehot2labels","repo":"MindSpore-scientific-2/code-3","repo_kind":"listed","path":"Three-Dimensional-Lip-Motion-Network-for-Text-Independent-Speaker-Recognition-master/3LMNet_ms.py","file_url":"https://github.com/MindSpore-scientific-2/code-3/blob/HEAD/Three-Dimensional-Lip-Motion-Network-for-Text-Independent-Speaker-Recognition-master/3LMNet_ms.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c1b2df7fb7a5f6f"}},{"code_sha256_prefix":"1ddbad7728b2f62f","entry":"read_file","repo":"octree-nn/octformer","repo_kind":"official","path":"datasets/modelnet40.py","file_url":"https://github.com/octree-nn/octformer/blob/HEAD/datasets/modelnet40.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ddbad7728b2f62f"}},{"code_sha256_prefix":"9a0dfd5a0f02da54","entry":"anet1","repo":"MindSpore-scientific-2/code-3","repo_kind":"listed","path":"Two-Layer-ReLU-Network-Analytically/anmin.py","file_url":"https://github.com/MindSpore-scientific-2/code-3/blob/HEAD/Two-Layer-ReLU-Network-Analytically/anmin.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9a0dfd5a0f02da54"}},{"code_sha256_prefix":"c3dc89be963fbdf9","entry":"color_distort","repo":"octree-nn/octformer","repo_kind":"official","path":"datasets/scannet.py","file_url":"https://github.com/octree-nn/octformer/blob/HEAD/datasets/scannet.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3dc89be963fbdf9"}},{"code_sha256_prefix":"fe0f3b2b7badf758","entry":"read_data","repo":"MindSpore-scientific-2/code-3","repo_kind":"listed","path":"The_RefinedWeb_Dataset_for_Falcon/eval_factuality_mindspore.py","file_url":"https://github.com/MindSpore-scientific-2/code-3/blob/HEAD/The_RefinedWeb_Dataset_for_Falcon/eval_factuality_mindspore.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe0f3b2b7badf758"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}