Papers › OneFormer3D: One Transformer for Unified Point Cloud Segmentation

OneFormer3D: One Transformer for Unified Point Cloud Segmentation

24 Nov 2023CVPR 2024 1arXiv:2311.14405archive 2025-07-28

Maxim Kolodiazhnyi, Anna Vorontsova, Anton Konushin, Danila Rukhovich

Semantic, instance, and panoptic segmentation of 3D point clouds have been addressed using task-specific models of distinct design. Thereby, the similarity of all segmentation tasks and the implicit relationship between them have not been utilized effectively. This paper presents a unified, simple, and effective model addressing all these tasks jointly. The model, named OneFormer3D, performs instance and semantic segmentation consistently, using a group of learnable kernels, where each kernel is responsible for generating a mask for either an instance or a semantic category. These kernels are trained with a transformer-based decoder with unified instance and semantic queries passed as an input. Such a design enables training a model end-to-end in a single run, so that it achieves top performance on all three segmentation tasks simultaneously. Specifically, our OneFormer3D ranks 1st and sets a new state-of-the-art (+2.1 mAP50) in the ScanNet test leaderboard. We also demonstrate the state-of-the-art results in semantic, instance, and panoptic segmentation of ScanNet (+21 PQ), ScanNet200 (+3.8 mAP50), and S3DIS (+0.8 mIoU) datasets.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

oneformer3d/oneformer3d mentioned on GitHubpytorchNOASSERTION report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

3D Instance Segmentation3D Object Detection3D Semantic SegmentationDecoderPanoptic SegmentationPoint Cloud SegmentationSegmentationSemantic Segmentation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
3D Instance Segmentation S3DIS OneFormer3D AP@50 75.8 #1 of 21 Archive leaderboard report
3D Instance Segmentation S3DIS OneFormer3D mAP 63.0 #1 of 21 Archive leaderboard report
3D Instance Segmentation S3DIS OneFormer3D mPrec 82.3 #1 of 21 Archive leaderboard report
3D Instance Segmentation S3DIS OneFormer3D mRec 74.1 #1 of 21 Archive leaderboard report
3D Instance Segmentation ScanNet(v2) OneFromer3D mAP 56.6 #4 of 32 Archive leaderboard report
3D Instance Segmentation ScanNet(v2) OneFromer3D mAP @ 50 80.1 #4 of 32 Archive leaderboard report
3D Instance Segmentation ScanNet(v2) OneFromer3D mAP@25 89.6 #4 of 32 Archive leaderboard report
3D Object Detection ScanNetV2 OneFormer3D mAP@0.25 76.9 #5 of 33 Archive leaderboard report
3D Object Detection ScanNetV2 OneFormer3D mAP@0.5 65.3 #5 of 33 Archive leaderboard report
3D Semantic Segmentation S3DIS OneFormer3D mIoU (6-Fold) 75.0 #1 of 6 Archive leaderboard report
3D Semantic Segmentation S3DIS OneFormer3D mIoU (Area-5) 72.4 #1 of 6 Archive leaderboard report
3D Semantic Segmentation ScanNet200 OneFormer3D val mIoU 30.1 #13 of 16 Archive leaderboard report
Panoptic Segmentation ScanNet OneFormer3D PQ 71.2 #1 of 4 Archive leaderboard report
Panoptic Segmentation ScanNet OneFormer3D PQ_st 86.1 #1 of 4 Archive leaderboard report
Panoptic Segmentation ScanNet OneFormer3D PQ_th 69.6 #1 of 4 Archive leaderboard report
Panoptic Segmentation ScanNetV2 OneFormer3D PQ 71.2 #1 of 5 Archive leaderboard report
Semantic Segmentation ScanNet OneFormer3D val mIoU 76.6 #11 of 45 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections