Papers › CSFNet: A Cosine Similarity Fusion Network for Real-Time RGB-X Semantic Segmentation...

CSFNet: A Cosine Similarity Fusion Network for Real-Time RGB-X Semantic Segmentation of Driving Scenes

1 Jul 2024arXiv:2407.01328archive 2025-07-28

Danial Qashqai, Emad Mousavian, Shahriar Baradaran Shokouhi, Sattar Mirzakuchaki

Semantic segmentation, as a crucial component of complex visual interpretation, plays a fundamental role in autonomous vehicle vision systems. Recent studies have significantly improved the accuracy of semantic segmentation by exploiting complementary information and developing multimodal methods. Despite the gains in accuracy, multimodal semantic segmentation methods suffer from high computational complexity and low inference speed. Therefore, it is a challenging task to implement multimodal methods in driving applications. To address this problem, we propose the Cosine Similarity Fusion Network (CSFNet) as a real-time RGB-X semantic segmentation model. Specifically, we design a Cosine Similarity Attention Fusion Module (CS-AFM) that effectively rectifies and fuses features of two modalities. The CS-AFM module leverages cross-modal similarity to achieve high generalization ability. By enhancing the fusion of cross-modal features at lower levels, CS-AFM paves the way for the use of a single-branch network at higher levels. Therefore, we use dual and single-branch architectures in an encoder, along with an efficient context module and a lightweight decoder for fast and accurate predictions. To verify the effectiveness of CSFNet, we use the Cityscapes, MFNet, and ZJU datasets for the RGB-D/T/P semantic segmentation. According to the results, CSFNet has competitive accuracy with state-of-the-art methods while being state-of-the-art in terms of speed among multimodal semantic segmentation models. It also achieves high efficiency due to its low parameter count and computational complexity. The source code for CSFNet will be available at https://github.com/Danial-Qashqai/CSFNet.

PaperPDFCode

Code

Danial-Qashqai/CSFNet officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Autonomous VehiclesImage SegmentationReal-Time Semantic SegmentationScene UnderstandingSegmentationSemantic SegmentationThermal Image Segmentation

1 archive task tag without a task page not shown.

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Real-Time Semantic Segmentation Cityscapes val CSFNet-2 Frame (fps) 72.3 (3090) #12 of 24 Archive leaderboard report
Real-Time Semantic Segmentation Cityscapes val CSFNet-2 mIoU 76.36 #12 of 24 Archive leaderboard report
Real-Time Semantic Segmentation Cityscapes val CSFNet-1 Frame (fps) 106.1 #18 of 24 Archive leaderboard report
Real-Time Semantic Segmentation Cityscapes val CSFNet-1 mIoU 74.73 #18 of 24 Archive leaderboard report
Semantic Segmentation Cityscapes val CSFNet-2 FPS 72.3 (3090) #69 of 99 Archive leaderboard report
Semantic Segmentation Cityscapes val CSFNet-2 mIoU 76.36 #69 of 99 Archive leaderboard report
Semantic Segmentation Cityscapes val CSFNet-1 FPS 106.1 (3090) #75 of 99 Archive leaderboard report
Semantic Segmentation Cityscapes val CSFNet-1 mIoU 74.73 #75 of 99 Archive leaderboard report
Semantic Segmentation ZJU-RGB-P CSFNet-2 Frame (fps) 75 (3090) #7 of 13 Archive leaderboard report
Semantic Segmentation ZJU-RGB-P CSFNet-2 mIoU 91.40 #7 of 13 Archive leaderboard report
Semantic Segmentation ZJU-RGB-P CSFNet-1 Frame (fps) 108.5 #8 of 13 Archive leaderboard report
Semantic Segmentation ZJU-RGB-P CSFNet-1 mIoU 90.85 #8 of 13 Archive leaderboard report
Thermal Image Segmentation MFN Dataset CSFNet-2 Frame (fps) 72.7 (3090) #5 of 55 Archive leaderboard report
Thermal Image Segmentation MFN Dataset CSFNet-2 mIOU 59.98 #5 of 55 Archive leaderboard report
Thermal Image Segmentation MFN Dataset CSFNet-1 Frame (fps) 106.3 (3090) #29 of 55 Archive leaderboard report
Thermal Image Segmentation MFN Dataset CSFNet-1 mIOU 56.05 #29 of 55 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

AttentionSPEEDSoftmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections