Papers › Beyond Appearances: Material Segmentation with Embedded Spectral Information from RGB-D imagery
Beyond Appearances: Material Segmentation with Embedded Spectral Information from RGB-D imagery
Fabian Perez, Hoover Rueda-Chacón
In the realm of computer vision material segmentation of natural scenes represents a challenge driven by the complex and diverse appearances of materials. Traditional approaches often rely on RGB images which can be deceptive given the variability in appearances due to different lighting conditions. Other methods that employ polarization or spectral imagery offer a more reliable material differentiation but their cost and accessibility restrict their everyday usage. In this work we propose a deep learning framework that bridges the gap between high-fidelity material segmentation and the practical constraints of data acquisition. Our approach leverages a training strategy that employs a paired RGBD-spectral data to incorporate spectral information directly within the neural network. This encoding process is facilitated by a Spectral Feature Mapper (SFM) layer a novel module that embeds unique spectral characteristics into the network thus enabling the network to infer materials from standard RGB-D images. Once trained the model allows to conduct material segmentation on widely available devices without the need for direct spectral data input. In addition we generate the 3D point cloud from the RGB-D image pair to provide a richer spatial context for scene understanding. Through simulations using available datasets and real experiments conducted with an iPad Pro our method demonstrates superior performance in material segmentation compared to other methods. Code is available at: https://github.com/Factral/Spectral-material-segmentation
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections