{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/insanet-intra-inter-spectral-attention","title":"INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection","arxiv_id":null,"date":"2024-02-10","proceeding":"journal 2024 2","authors":["S. Lee","T. Kim","J. Shin","N. Kim","Y. Choi"],"abstract":"Pedestrian detection is a critical task for safety-critical systems, but detecting pedestrians is challenging in low-light and adverse weather conditions. Thermal images can be used to improve robustness by providing complementary information to RGB images. Previous studies have shown that multi-modal feature fusion using convolution operation can be effective, but such methods rely solely on local feature correlations, which can degrade the performance capabilities. To address this issue, we propose an attention-based novel fusion network, referred to as INSANet (INtra-INter Spectral Attention Network), that captures global intra- and inter-information. It consists of intra- and inter-spectral attention blocks that allow the model to learn mutual spectral relationships. Additionally, we identified an imbalance in the multispectral dataset caused by several factors and designed an augmentation strategy that mitigates concentrated distributions and enables the model to learn the diverse locations of pedestrians. Extensive experiments demonstrate the effectiveness of the proposed methods, which achieve state-of-the-art performance on the KAIST dataset and LLVIP dataset. Finally, we conduct a regional performance evaluation to demonstrate the effectiveness of our proposed network in various regions.","url_abs":"https://www.mdpi.com/1424-8220/24/4/1168","url_pdf":"https://www.mdpi.com/1424-8220/24/4/1168/pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"insanet-intra-inter-spectral-attention","repo_url":"https://github.com/sejong-rcv/INSANet","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multispectral-object-detection","task_name":"Multispectral Object Detection"},{"task_slug":"pedestrian-detection","task_name":"Pedestrian Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/multispectral-object-detection-on-kaist","task":"Multispectral Object Detection","dataset":"KAIST Multispectral Pedestrian Detection Benchmark","model":"INSANet","rank_in_archive_order":3,"of":17,"metrics":{"All Miss Rate":"26.08","Reasonable Miss Rate":"5.50"},"uses_additional_data":false},{"leaderboard":"/sota/pedestrian-detection-on-llvip","task":"Pedestrian Detection","dataset":"LLVIP","model":"INSANet","rank_in_archive_order":15,"of":15,"metrics":{"log average miss rate":"4.43%"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}