Papers › IAM: Enhancing RGB-D Instance Segmentation with New Benchmarks

IAM: Enhancing RGB-D Instance Segmentation with New Benchmarks

3 Jan 2025arXiv:2501.01685archive 2025-07-28

Aecheon Jung, Soyun Choi, Junhong Min, Sungeun Hong

Image segmentation is a vital task for providing human assistance and enhancing autonomy in our daily lives. In particular, RGB-D segmentation-leveraging both visual and depth cues-has attracted increasing attention as it promises richer scene understanding than RGB-only methods. However, most existing efforts have primarily focused on semantic segmentation and thus leave a critical gap. There is a relative scarcity of instance-level RGB-D segmentation datasets, which restricts current methods to broad category distinctions rather than fully capturing the fine-grained details required for recognizing individual objects. To bridge this gap, we introduce three RGB-D instance segmentation benchmarks, distinguished at the instance level. These datasets are versatile, supporting a wide range of applications from indoor navigation to robotic manipulation. In addition, we present an extensive evaluation of various baseline models on these benchmarks. This comprehensive analysis identifies both their strengths and shortcomings, guiding future work toward more robust, generalizable solutions. Finally, we propose a simple yet effective method for RGB-D data integration. Extensive evaluations affirm the effectiveness of our approach, offering a robust framework for advancing toward more nuanced scene understanding.

PaperPDFCode

Code

aim-skku/box-is officialmentioned in papermentioned on GitHub report
aim-skku/nyudv2-is officialmentioned in papermentioned on GitHub report
aim-skku/sun-rgbd-is officialmentioned in papermentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Data IntegrationImage SegmentationInstance SegmentationRGB-D Instance SegmentationScene UnderstandingSegmentationSemantic Segmentation

Datasets

Introduced by this paper, per the archive.

Box-ISNYUDv2-ISSUN-RGBD-IS

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Instance Segmentation Box-IS IAM + SOLQ mask AP 83.7 #1 of 1 Archive leaderboard report
Instance Segmentation NYUDv2-IS IAM + SOLQ mask AP 35.8 #1 of 2 Archive leaderboard report
Instance Segmentation NYUDv2-IS IAM + DETR mask AP 32.3 #2 of 2 Archive leaderboard report
Instance Segmentation SUN-RGBD-IS IAM + SOLQ mask AP 25.7 #1 of 2 Archive leaderboard report
Instance Segmentation SUN-RGBD-IS IAM + DETR mask AP 22.9 #2 of 2 Archive leaderboard report
RGB-D Instance Segmentation Box-IS IAM + SOLQ mask AP 83.7 #1 of 1 Archive leaderboard report
RGB-D Instance Segmentation NYUDv2-IS IAM + SOLQ mask AP 35.8 #1 of 2 Archive leaderboard report
RGB-D Instance Segmentation NYUDv2-IS IAM + DETR mask AP 32.3 #2 of 2 Archive leaderboard report
RGB-D Instance Segmentation SUN-RGBD-IS IAM + SOLQ mask AP 25.7 #1 of 2 Archive leaderboard report
RGB-D Instance Segmentation SUN-RGBD-IS IAM + DETR mask AP 22.9 #2 of 2 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

AttentionSoftmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections