Papers › RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes
RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes
Yuxiang Sun, Weixun Zuo, Ming Liu
Semantic segmentation is a fundamental capability for autonomous vehicles. With the advancements of deep learning technologies, many effective semantic segmentation networks have been proposed in recent years. However, most of them are designed using RGB images from visible cameras. The quality of RGB images is prone to be degraded under unsatisfied lighting conditions, such as darkness and glares of oncoming headlights, which imposes critical challenges for the networks that use only RGB images. Different from visible cameras, thermal imaging cameras generate images using thermal radiations. They are able to see under various lighting conditions. In order to enable robust and accurate semantic segmentation for autonomous vehicles, we take the advantage of thermal images and fuse both the RGB and thermal information in a novel deep neural network. The main innovation of this letter is the architecture of the proposed network. We adopt the encoder–decoder design concept. ResNet is employed for feature extraction and a new decoder is developed to restore the feature map resolution. The experimental results prove that our network outperforms the state of the arts.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Semantic Segmentation | GAMUS | RTFNet | mIoU | 58.26 | #4 of 6 | Archive leaderboard | report |
| Semantic Segmentation | SYN-UDTIRI | RTFNet | IoU | 90.50 | #8 of 10 | Archive leaderboard | report |
| Semantic Segmentation | Synthetic Bathing Perception | RTFNet | mIoU | 87.49 | #3 of 5 | Archive leaderboard | report |
| Thermal Image Segmentation | KP day-night | RTFNet | mIoU | 28.7 | #4 of 5 | Archive leaderboard | report |
| Thermal Image Segmentation | MFN Dataset | RTFNet | mIOU | 53.2 | #39 of 55 | Archive leaderboard | report |
| Thermal Image Segmentation | Noisy RS RGB-T Dataset | RTFNet | mIoU | 48.5 | #5 of 6 | Archive leaderboard | report |
| Thermal Image Segmentation | PST900 | RTFNet | mIoU | 57.6 | #19 of 22 | Archive leaderboard | report |
| Thermal Image Segmentation | RGB-T-Glass-Segmentation | RTFNet | MAE | 0.058 | #14 of 22 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections