Browse State-of-the-Art › Multispectral Object Detection
Multispectral Object Detection
26 papers with code · 4 benchmarks · 4 datasets archive 2025-07-28
Only using RGB cameras for automatic outdoor scene analysis is challenging when, for example, facing insufficient illumination or adverse weather. To improve the recognition reliability, multispectral systems add additional cameras (e.g. infra-red) and perform object detection from multispectral data. Although multispectral scene analysis with deep learning has be shown to have a great potential, there are still many open research questions and it has not been widely deployed in industrial contexts.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
4 leaderboard tables shown for this task, 4 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| FLIR (18 rows) | MMPedestron | When Pedestrian Detection Meets Multi-Modal Learning: Generalist... | code | — | Compare |
| KAIST Multispectral Pedestrian Detection Benchmark (17 rows) | RSDet | Removal then Selection: A Coarse-to-Fine Fusion Perspective for... | code | — | Compare |
| NII-CU MAPD (2 rows) | YOLOv3-4‐channel | Deep learning with RGB and thermal images onboard a drone for... | — | — | Compare |
| LLVIP (1 row) | CFT | Cross-Modality Fusion Transformer for Multispectral Object Detection | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
4 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 39 · with a code link: 26 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 39 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 39 of 39, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
14 Nov 2014 51 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)Convolutional networks are powerful visual models that yield hierarchies of features.
-
28 Jun 2023 2 repositories listedIn C²Former, we design an Inter-modality Cross-Attention (ICA) module to obtain the calibrated and complementary features by learning the cross-attention relationship between the RGB and IR modality.
-
7 Aug 2020 2 repositories listedCompared with traditional pedestrian detection, we find multispectral pedestrian detection suffers from modality imbalance problems which will hinder the optimization process of dual-modality network and depress the…
-
8 Nov 2016 2 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 7 harvested samples) · 5 pointer-only (licence)Multispectral pedestrian detection is essential for around-the-clock applications, e. g., surveillance and autonomous driving.
-
17 Jun 2025 1 repository listedTo address these, based on the YOLOv11 framework, we present YOLOv11-RGBT, a new comprehensive multimodal object detection framework.
-
21 May 2025 1 repository listedMultispectral object detection aims to leverage complementary information from visible (RGB) and infrared (IR) modalities to enable robust performance under diverse environmental conditions.
-
14 Jul 2024 1 repository listedWith multi-modal joint training, our model achieves state-of-the-art performance on a wide range of pedestrian detection benchmarks, surpassing leading models tailored for specific sensor modality.
-
25 May 2024 1 repository listedIn this paper, we address this issue by improving the performance of efficient single-branch structures.
-
29 Apr 2024 1 repository listedMultimodal learning is a common way to leverage these modalities, where multiple modality-specific encoders and a fusion module are used to improve performance.
-
26 Apr 2024 1 repository listedSemantic analysis on visible (RGB) and infrared (IR) images has gained attention for its ability to be more accurate and robust under low-illumination and complex weather conditions.
-
25 Apr 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 7 pointer-only (licence)Furthermore, we introduce the Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment detection accuracy in adverse weather conditions.
-
10 Feb 2024 1 repository listedExtensive experiments demonstrate the effectiveness of the proposed methods, which achieve state-of-the-art performance on the KAIST dataset and LLVIP dataset.
-
19 Jan 2024 1 repository listedSpecifically, following this perspective, we design a Redundant Spectrum Removal module to remove interfering information within each modality coarsely and a Dynamic Feature Selection module to finely select the desired…
-
30 Oct 2023 1 repository listedMultimodal deep sensor fusion has the potential to enable autonomous vehicles to visually understand their surrounding environments in all weather conditions.
-
15 Aug 2023 1 repository listedEffective feature fusion of multispectral images plays a crucial role in multi-spectral object detection.
-
26 May 2023 1 repository listedDifferent from them, we comprehensively analyze the impacts of false positives on the detection performance and find that enhancing feature contrast can significantly reduce these false positives.
-
21 Mar 2022 1 repository listedMultispectral pedestrian detection is an important and valuable task in many applications, which could provide a more accurate and reliable pedestrian detection result by using the complementary visual information from…
-
9 Mar 2022 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)Pixel-wise semantic segmentation of RGB images can be advanced by exploiting complementary features from the supplementary modality (X-modality).
-
21 Dec 2021 1 repository listedHowever, these methods inherently suffer from the ill-posedness of the joint reconstruction problem.
-
30 Oct 2021 1 repository listedMultispectral image pairs can provide the combined information, making object detection applications more reliable and robust in the open world.
-
24 Aug 2021 1 repository listedIt is very challenging for various visual tasks such as image fusion, pedestrian detection and image-to-image translation in low light conditions due to the loss of effective target areas.
-
26 Jul 2021 1 repository listedIn this letter, we tackle multispectral pedestrian detection, where all input data are not paired.
-
3 Jan 2021 1 repository listedMultispectral image pairs can provide complementary visual information, making pedestrian detection systems more robust and reliable.
-
26 Sep 2020 1 repository listedMultispectral images (e. g. visible and infrared) may be particularly useful when detecting objects with the same model in different environments (e. g. day/night outdoor scenes).
-
27 Nov 2018 1 repository listedWeakly supervised semantic segmentation with only image-level labels saves large human effort to annotate pixel-level labels.
-
14 Aug 2018 1 repository listedTo narrow this gap, we propose a network fusion architecture, which consists of a multispectral proposal network to generate pedestrian proposals, and a subsequent multispectral classification network to distinguish…
-
Optimizing Multispectral Object Detection: A Bag of Tricks and Comprehensive Benchmarks27 Nov 2024 0 repositories listedMultispectral object detection, utilizing RGB and TIR (thermal infrared) modalities, is widely recognized as a challenging task.
-
27 Sep 2024 0 repositories listedObject detection on images can find benefit from coupling multiple spectra, each presenting specific useful features.
-
Surveying You Only Look Once (YOLO) Multispectral Object Detection Advancements, Applications And Challenges3 Sep 2024 0 repositories listedMultispectral imaging and deep learning have emerged as powerful tools supporting diverse use cases from autonomous vehicles, to agriculture, infrastructure monitoring and environmental assessment.
-
RGB-T Object Detection via Group Shuffled Multi-receptive Attention and Multi-modal Supervision29 May 2024 0 repositories listedMultispectral object detection, utilizing both visible (RGB) and thermal infrared (T) modals, has garnered significant attention for its robust performance across diverse weather and lighting conditions.
-
A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)18 Feb 2024 0 repositories listed
-
18 Jun 2023 0 repositories listed
-
28 Sep 2022 0 repositories listed
-
31 May 2022 0 repositories listed
-
A Comparison of Deep Saliency Map Generators on Multispectral Data in Object Detection26 Aug 2021 0 repositories listed
-
Multispectral Object Detection with Deep Learning5 Feb 2021 0 repositories listed
-
9 Jan 2019 0 repositories listed
-
14 Mar 2018 0 repositories listed
-
27 Feb 2018 0 repositories listed
Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections