{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sequential-ensembling-for-semantic","title":"Sequential Ensembling for Semantic Segmentation","arxiv_id":"2210.05387","date":"2022-10-08","proceeding":null,"authors":["Rawal Khirodkar","Brandon Smith","Siddhartha Chandra","Amit Agrawal","Antonio Criminisi"],"abstract":"Ensemble approaches for deep-learning-based semantic segmentation remain insufficiently explored despite the proliferation of competitive benchmarks and downstream applications. In this work, we explore and benchmark the popular ensembling approach of combining predictions of multiple, independently-trained, state-of-the-art models at test time on popular datasets. Furthermore, we propose a novel method inspired by boosting to sequentially ensemble networks that significantly outperforms the naive ensemble baseline. Our approach trains a cascade of models conditioned on class probabilities predicted by the previous model as an additional input. A key benefit of this approach is that it allows for dynamic computation offloading, which helps deploy models on mobile devices. Our proposed novel ADaptive modulatiON (ADON) block allows spatial feature modulation at various layers using previous-stage probabilities. Our approach does not require sophisticated sample selection strategies during training and works with multiple neural architectures. We significantly improve over the naive ensemble baseline on challenging datasets such as Cityscapes, ADE-20K, COCO-Stuff, and PASCAL-Context and set a new state-of-the-art.","url_abs":"https://arxiv.org/abs/2210.05387v1","url_pdf":"https://arxiv.org/pdf/2210.05387v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"Sequential Ensemble (SegFormer)","rank_in_archive_order":70,"of":235,"metrics":{"Params (M)":"216.3","Validation mIoU":"54"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"Sequential Ensemble (DeepLabv3+)","rank_in_archive_order":171,"of":235,"metrics":{"Validation mIoU":"46.8"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes-val","task":"Semantic Segmentation","dataset":"Cityscapes val","model":"Sequential Ensemble (MiT-B5 + HRNet)","rank_in_archive_order":12,"of":99,"metrics":{"mIoU":"84.8"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-pascal-context","task":"Semantic Segmentation","dataset":"PASCAL Context","model":"Sequential Ensemble (Segformer + HRNet)","rank_in_archive_order":11,"of":66,"metrics":{"mIoU":"62.1"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}