Papers › SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences

SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences

27 Mar 2021CVPR 2021 1arXiv:2103.14898archive 2025-07-28

Shun-Cheng Wu, Johanna Wald, Keisuke Tateno, Nassir Navab, Federico Tombari

Scene graphs are a compact and explicit representation successfully used in a variety of 2D scene understanding tasks. This work proposes a method to incrementally build up semantic scene graphs from a 3D environment given a sequence of RGB-D frames. To this end, we aggregate PointNet features from primitive scene components by means of a graph neural network. We also propose a novel attention mechanism well suited for partial and missing graph data present in such an incremental reconstruction scenario. Although our proposed method is designed to run on submaps of the scene, we show it also transfers to entire 3D scenes. Experiments show that our approach outperforms 3D scene graph prediction methods by a large margin and its accuracy is on par with other 3D semantic and panoptic segmentation methods while running at 35 Hz.

PaperPDFConference PDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

ShunChengWu/3DSSG officialmentioned on GitHubpytorchNOASSERTION report
ShunChengWu/SceneGraphFusion officialmentioned on GitHubBSD-2-Clause report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

3D Object Classification3d scene graph generationGraph Neural NetworkPanoptic SegmentationPredicate ClassificationScene Graph GenerationScene Understanding

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
3D Object Classification 3R-Scan SceneGraphFusion Top-10 Accuracy 0.8 #1 of 2 Archive leaderboard report
3D Object Classification 3R-Scan SceneGraphFusion Top-5 Accuracy 0.7 #1 of 2 Archive leaderboard report
3D Object Classification 3R-Scan 3DSSG [Wald2020_3dssg] Top-10 Accuracy 0.78 #2 of 2 Archive leaderboard report
3D Object Classification 3R-Scan 3DSSG [Wald2020_3dssg] Top-5 Accuracy 0.68 #2 of 2 Archive leaderboard report
Panoptic Segmentation ScanNet SceneGraphFusion PQ 31.5 #4 of 4 Archive leaderboard report
Panoptic Segmentation ScanNet SceneGraphFusion PQ_st 43.4 #4 of 4 Archive leaderboard report
Panoptic Segmentation ScanNet SceneGraphFusion PQ_th 30.2 #4 of 4 Archive leaderboard report
Panoptic Segmentation ScanNetV2 SceneGraphFusion (NN mapping) PQ 31.5 #5 of 5 Archive leaderboard report
Panoptic Segmentation ScanNetV2 SceneGraphFusion (NN mapping) Params (M) 2.9 #5 of 5 Archive leaderboard report
Panoptic Segmentation ScanNetV2 SceneGraphFusion (NN mapping) RQ 42.2 #5 of 5 Archive leaderboard report
Panoptic Segmentation ScanNetV2 SceneGraphFusion (NN mapping) SQ 72.9 #5 of 5 Archive leaderboard report
Scene Graph Generation 3R-Scan SceneGraphFusion Top-5 Accuracy 0.87 #1 of 2 Archive leaderboard report
Scene Graph Generation 3R-Scan 3DSSG [Wald2020_3dssg] Top-5 Accuracy 0.66 #2 of 2 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections