{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/acnet-attention-based-network-to-exploit","title":"ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation","arxiv_id":"1905.10089","date":"2019-05-24","proceeding":null,"authors":["Xinxin Hu","Kailun Yang","Lei Fei","Kaiwei Wang"],"abstract":"Compared to RGB semantic segmentation, RGBD semantic segmentation can achieve better performance by taking depth information into consideration. However, it is still problematic for contemporary segmenters to effectively exploit RGBD information since the feature distributions of RGB and depth (D) images vary significantly in different scenes. In this paper, we propose an Attention Complementary Network (ACNet) that selectively gathers features from RGB and depth branches. The main contributions lie in the Attention Complementary Module (ACM) and the architecture with three parallel branches. More precisely, ACM is a channel attention-based module that extracts weighted features from RGB and depth branches. The architecture preserves the inference of the original RGB and depth branches, and enables the fusion branch at the same time. Based on the above structures, ACNet is capable of exploiting more high-quality features from different channels. We evaluate our model on SUN-RGBD and NYUDv2 datasets, and prove that our model outperforms state-of-the-art methods. In particular, a mIoU score of 48.3\\% on NYUDv2 test set is achieved with ResNet50. We will release our source code based on PyTorch and the trained segmentation model at https://github.com/anheidelonghu/ACNet.","url_abs":"https://arxiv.org/abs/1905.10089v1","url_pdf":"https://arxiv.org/pdf/1905.10089v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"acnet-attention-based-network-to-exploit","repo_url":"https://github.com/anheidelonghu/ACNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"RGBD Semantic Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"thermal-image-segmentation","task_name":"Thermal Image Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-segmentation-on-kitti-360","task":"Semantic Segmentation","dataset":"KITTI-360","model":"ACNet (ResNet50)","rank_in_archive_order":7,"of":17,"metrics":{"mIoU":"61.57"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-nyu-depth-v2","task":"Semantic Segmentation","dataset":"NYU Depth v2","model":"ACNet","rank_in_archive_order":78,"of":121,"metrics":{"Mean IoU":"48.3%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-sun-rgbd","task":"Semantic Segmentation","dataset":"SUN-RGBD","model":"CMX (B4)","rank_in_archive_order":33,"of":44,"metrics":{"Mean IoU":"48.1%"},"uses_additional_data":true},{"leaderboard":"/sota/semantic-segmentation-on-thud-robotic-dataset","task":"Semantic Segmentation","dataset":"THUD Robotic Dataset","model":"ACNet","rank_in_archive_order":4,"of":4,"metrics":{"mIoU":"74.83"},"uses_additional_data":false},{"leaderboard":"/sota/thermal-image-segmentation-on-mfn-dataset","task":"Thermal Image Segmentation","dataset":"MFN Dataset","model":"ACNet","rank_in_archive_order":45,"of":55,"metrics":{"mIOU":"46.3"},"uses_additional_data":false},{"leaderboard":"/sota/thermal-image-segmentation-on-pst900","task":"Thermal Image Segmentation","dataset":"PST900","model":"ACNet","rank_in_archive_order":17,"of":22,"metrics":{"mIoU":"71.81"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1905.10089","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}