{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/swin-umamba-mamba-based-unet-with-imagenet","title":"Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining","arxiv_id":"2402.03302","date":"2024-02-05","proceeding":null,"authors":["Jiarun Liu","Hao Yang","Hong-Yu Zhou","Yan Xi","Lequan Yu","Yizhou Yu","Yong Liang","Guangming Shi","Shaoting Zhang","Hairong Zheng","Shanshan Wang"],"abstract":"Accurate medical image segmentation demands the integration of multi-scale information, spanning from local features to global dependencies. However, it is challenging for existing methods to model long-range global information, where convolutional neural networks (CNNs) are constrained by their local receptive fields, and vision transformers (ViTs) suffer from high quadratic complexity of their attention mechanism. Recently, Mamba-based models have gained great attention for their impressive ability in long sequence modeling. Several studies have demonstrated that these models can outperform popular vision models in various tasks, offering higher accuracy, lower memory consumption, and less computational burden. However, existing Mamba-based models are mostly trained from scratch and do not explore the power of pretraining, which has been proven to be quite effective for data-efficient medical image analysis. This paper introduces a novel Mamba-based model, Swin-UMamba, designed specifically for medical image segmentation tasks, leveraging the advantages of ImageNet-based pretraining. Our experimental results reveal the vital role of ImageNet-based training in enhancing the performance of Mamba-based models. Swin-UMamba demonstrates superior performance with a large margin compared to CNNs, ViTs, and latest Mamba-based models. Notably, on AbdomenMRI, Encoscopy, and Microscopy datasets, Swin-UMamba outperforms its closest counterpart U-Mamba_Enc by an average score of 2.72%.","url_abs":"https://arxiv.org/abs/2402.03302v2","url_pdf":"https://arxiv.org/pdf/2402.03302v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"swin-umamba-mamba-based-unet-with-imagenet","repo_url":"https://github.com/jiarunliu/swin-umamba","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"medical-image-analysis","task_name":"Medical Image Analysis"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.03302","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03302"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiarunliu/swin-umamba","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a2c8b38ba9393780","entry":"convert_labels_back_to_BraTS","repo":"jiarunliu/swin-umamba","repo_kind":"official","path":"swin_umamba/nnunetv2/dataset_conversion/Dataset137_BraTS21.py","file_url":"https://github.com/jiarunliu/swin-umamba/blob/HEAD/swin_umamba/nnunetv2/dataset_conversion/Dataset137_BraTS21.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2c8b38ba9393780"}},{"code_sha256_prefix":"517ad51ee69e8a2d","entry":"copy_files","repo":"jiarunliu/swin-umamba","repo_kind":"official","path":"swin_umamba/nnunetv2/dataset_conversion/Dataset027_ACDC.py","file_url":"https://github.com/jiarunliu/swin-umamba/blob/HEAD/swin_umamba/nnunetv2/dataset_conversion/Dataset027_ACDC.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"517ad51ee69e8a2d"}},{"code_sha256_prefix":"c66e8feb67225a1b","entry":"copy_files","repo":"jiarunliu/swin-umamba","repo_kind":"official","path":"swin_umamba/nnunetv2/dataset_conversion/Dataset115_EMIDEC.py","file_url":"https://github.com/jiarunliu/swin-umamba/blob/HEAD/swin_umamba/nnunetv2/dataset_conversion/Dataset115_EMIDEC.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c66e8feb67225a1b"}},{"code_sha256_prefix":"2a83a2bc5120febe","entry":"get_vendor_split","repo":"jiarunliu/swin-umamba","repo_kind":"official","path":"swin_umamba/nnunetv2/dataset_conversion/Dataset114_MNMs.py","file_url":"https://github.com/jiarunliu/swin-umamba/blob/HEAD/swin_umamba/nnunetv2/dataset_conversion/Dataset114_MNMs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a83a2bc5120febe"}},{"code_sha256_prefix":"86ec2ccc2984b8bf","entry":"merge","repo":"jiarunliu/swin-umamba","repo_kind":"official","path":"swin_umamba/nnunetv2/batch_running/generate_lsf_runs_customDecathlon.py","file_url":"https://github.com/jiarunliu/swin-umamba/blob/HEAD/swin_umamba/nnunetv2/batch_running/generate_lsf_runs_customDecathlon.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"86ec2ccc2984b8bf"}},{"code_sha256_prefix":"882441705cf75b1d","entry":"read_csv","repo":"jiarunliu/swin-umamba","repo_kind":"official","path":"swin_umamba/nnunetv2/dataset_conversion/Dataset114_MNMs.py","file_url":"https://github.com/jiarunliu/swin-umamba/blob/HEAD/swin_umamba/nnunetv2/dataset_conversion/Dataset114_MNMs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"882441705cf75b1d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}