{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mrfp-learning-generalizable-semantic","title":"MRFP: Learning Generalizable Semantic Segmentation from Sim-2-Real with Multi-Resolution Feature Perturbation","arxiv_id":"2311.18331","date":"2023-11-30","proceeding":"CVPR 2024 1","authors":["Sumanth Udupa","Prajwal Gurunath","Aniruddh Sikdar","Suresh Sundaram"],"abstract":"Deep neural networks have shown exemplary performance on semantic scene understanding tasks on source domains, but due to the absence of style diversity during training, enhancing performance on unseen target domains using only single source domain data remains a challenging task. Generation of simulated data is a feasible alternative to retrieving large style-diverse real-world datasets as it is a cumbersome and budget-intensive process. However, the large domain-specfic inconsistencies between simulated and real-world data pose a significant generalization challenge in semantic segmentation. In this work, to alleviate this problem, we propose a novel MultiResolution Feature Perturbation (MRFP) technique to randomize domain-specific fine-grained features and perturb style of coarse features. Our experimental results on various urban-scene segmentation datasets clearly indicate that, along with the perturbation of style-information, perturbation of fine-feature components is paramount to learn domain invariant robust feature maps for semantic segmentation models. MRFP is a simple and computationally efficient, transferable module with no additional learnable parameters or objective functions, that helps state-of-the-art deep neural networks to learn robust domain invariant features for simulation-to-real semantic segmentation.","url_abs":"https://arxiv.org/abs/2311.18331v2","url_pdf":"https://arxiv.org/pdf/2311.18331v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mrfp-learning-generalizable-semantic","repo_url":"https://github.com/airl-iisc/MRFP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"2d-semantic-segmentation","task_name":"2D Semantic Segmentation"},{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"autonomous-vehicles","task_name":"Autonomous Vehicles"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"style-generalization","task_name":"Style Generalization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-segmentation-on-bdd100k-val","task":"Semantic Segmentation","dataset":"BDD100K val","model":"MRFP+(Ours) Resnet50","rank_in_archive_order":10,"of":24,"metrics":{"mIoU":"39.55"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-bdd100k-val","task":"Semantic Segmentation","dataset":"BDD100K val","model":"Resnet50","rank_in_archive_order":11,"of":24,"metrics":{"mIoU":"31.44"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes-val","task":"Semantic Segmentation","dataset":"Cityscapes val","model":"MRFP+(Ours) Resnet50","rank_in_archive_order":97,"of":99,"metrics":{"mIoU":"42.4"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-cityscapes-val","task":"Semantic Segmentation","dataset":"Cityscapes val","model":"Resnet50","rank_in_archive_order":98,"of":99,"metrics":{"mIoU":"34.66"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-mapillary-val","task":"Semantic Segmentation","dataset":"Mapillary val","model":"MRFP+(Ours) Resnet50","rank_in_archive_order":6,"of":8,"metrics":{"mIoU":"44.93"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-mapillary-val","task":"Semantic Segmentation","dataset":"Mapillary val","model":"Resnet50","rank_in_archive_order":8,"of":8,"metrics":{"mIoU":"32.93"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-synthia","task":"Semantic Segmentation","dataset":"SYNTHIA","model":"MRFP+(Ours) Resnet50","rank_in_archive_order":2,"of":3,"metrics":{"mIoU":"30.22"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-synthia","task":"Semantic Segmentation","dataset":"SYNTHIA","model":"Resnet50","rank_in_archive_order":3,"of":3,"metrics":{"mIoU":"25.84"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}