Papers › Learning Selective Self-Mutual Attention for RGB-D Saliency Detection

Learning Selective Self-Mutual Attention for RGB-D Saliency Detection

1 Jun 2020CVPR 2020 6archive 2025-07-28

Nian Liu, Ni Zhang, Junwei Han

Saliency detection on RGB-D images is receiving more and more research interests recently. Previous models adopt the early fusion or the result fusion scheme to fuse the input RGB and depth data or their saliency maps, which incur the problem of distribution gap or information loss. Some other models use the feature fusion scheme but are limited by the linear feature fusion methods. In this paper, we propose to fuse attention learned in both modalities. Inspired by the Non-local model, we integrate the self-attention and each other's attention to propagate long-range contextual dependencies, thus incorporating multi-modal information to learn attention and propagate contexts more accurately. Considering the reliability of the other modality's attention, we further propose a selection attention to weight the newly added attention term. We embed the proposed attention module in a two-stream CNN for RGB-D saliency detection. Furthermore, we also propose a residual fusion module to fuse the depth decoder features into the RGB stream. Experimental results on seven benchmark datasets demonstrate the effectiveness of the proposed model components and our final saliency model. Our code and saliency maps are available at https://github.com/nnizhang/S2MA.

Paper PDF Code

Code

nnizhang/S2MA officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Decoder RGB-D Salient Object Detection Saliency Detection

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
RGB-D Salient Object Detection	NJU2K	S2MA	Average MAE	0.053	#21 of 27	Archive leaderboard	report
RGB-D Salient Object Detection	NJU2K	S2MA	S-Measure	89.4	#21 of 27	Archive leaderboard	report
RGB-D Salient Object Detection	NJU2K	S2MA	max E-Measure	92.7	#21 of 27	Archive leaderboard	report
RGB-D Salient Object Detection	NJU2K	S2MA	max F-Measure	88.9	#21 of 27	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections