{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transfuse-fusing-transformers-and-cnns-for","title":"TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation","arxiv_id":"2102.08005","date":"2021-02-16","proceeding":null,"authors":["Yundong Zhang","Huiye Liu","Qiang Hu"],"abstract":"Medical image segmentation - the prerequisite of numerous clinical needs - has been significantly prospered by recent advances in convolutional neural networks (CNNs). However, it exhibits general limitations on modeling explicit long-range relation, and existing cures, resorting to building deep encoders along with aggressive downsampling operations, leads to redundant deepened networks and loss of localized details. Hence, the segmentation task awaits a better solution to improve the efficiency of modeling global contexts while maintaining a strong grasp of low-level details. In this paper, we propose a novel parallel-in-branch architecture, TransFuse, to address this challenge. TransFuse combines Transformers and CNNs in a parallel style, where both global dependency and low-level spatial details can be efficiently captured in a much shallower manner. Besides, a novel fusion technique - BiFusion module is created to efficiently fuse the multi-level features from both branches. Extensive experiments demonstrate that TransFuse achieves the newest state-of-the-art results on both 2D and 3D medical image sets including polyp, skin lesion, hip, and prostate segmentation, with significant parameter decrease and inference speed improvement.","url_abs":"https://arxiv.org/abs/2102.08005v2","url_pdf":"https://arxiv.org/pdf/2102.08005v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transfuse-fusing-transformers-and-cnns-for","repo_url":"https://github.com/Rayicer/TransFuse","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/medical-image-segmentation-on-cvc-clinicdb","task":"Medical Image Segmentation","dataset":"CVC-ClinicDB","model":"TransFuse-L","rank_in_archive_order":27,"of":48,"metrics":{"mean Dice":"0.934"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-cvc-clinicdb","task":"Medical Image Segmentation","dataset":"CVC-ClinicDB","model":"TransFuse-S","rank_in_archive_order":36,"of":48,"metrics":{"mean Dice":"0.918"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-cvc-colondb","task":"Medical Image Segmentation","dataset":"CVC-ColonDB","model":"TransFuse-S","rank_in_archive_order":19,"of":25,"metrics":{"mIoU":"0.696","mean Dice":"0.773"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-cvc-colondb","task":"Medical Image Segmentation","dataset":"CVC-ColonDB","model":"TransFuse-L","rank_in_archive_order":23,"of":25,"metrics":{"mIoU":"0.676","mean Dice":"0.744"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-etis","task":"Medical Image Segmentation","dataset":"ETIS-LARIBPOLYPDB","model":"TransFuse-L","rank_in_archive_order":17,"of":25,"metrics":{"mIoU":"0.661","mean Dice":"0.737"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-etis","task":"Medical Image Segmentation","dataset":"ETIS-LARIBPOLYPDB","model":"TransFuse-S","rank_in_archive_order":18,"of":25,"metrics":{"mIoU":"0.659","mean Dice":"0.733"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-kvasir-seg","task":"Medical Image Segmentation","dataset":"Kvasir-SEG","model":"TransFuse-L","rank_in_archive_order":26,"of":58,"metrics":{"mIoU":"0.868","mean Dice":"0.918"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-kvasir-seg","task":"Medical Image Segmentation","dataset":"Kvasir-SEG","model":"TransFuse-S","rank_in_archive_order":27,"of":58,"metrics":{"mIoU":"0.868","mean Dice":"0.918"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.08005","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}