{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/medical-image-segmentation-using-squeeze-and","title":"Medical Image Segmentation Using Squeeze-and-Expansion Transformers","arxiv_id":"2105.09511","date":"2021-05-20","proceeding":null,"authors":["Shaohua Li","Xiuchao Sui","Xiangde Luo","Xinxing Xu","Yong liu","Rick Goh"],"abstract":"Medical image segmentation is important for computer-aided diagnosis. Good segmentation demands the model to see the big picture and fine details simultaneously, i.e., to learn image features that incorporate large context while keep high spatial resolutions. To approach this goal, the most widely used methods -- U-Net and variants, extract and fuse multi-scale features. However, the fused features still have small \"effective receptive fields\" with a focus on local image cues, limiting their performance. In this work, we propose Segtran, an alternative segmentation framework based on transformers, which have unlimited \"effective receptive fields\" even at high feature resolutions. The core of Segtran is a novel Squeeze-and-Expansion transformer: a squeezed attention block regularizes the self attention of transformers, and an expansion block learns diversified representations. Additionally, we propose a new positional encoding scheme for transformers, imposing a continuity inductive bias for images. Experiments were performed on 2D and 3D medical image segmentation tasks: optic disc/cup segmentation in fundus images (REFUGE'20 challenge), polyp segmentation in colonoscopy images, and brain tumor segmentation in MRI scans (BraTS'19 challenge). Compared with representative existing methods, Segtran consistently achieved the highest segmentation accuracy, and exhibited good cross-domain generalization capabilities. The source code of Segtran is released at https://github.com/askerlee/segtran.","url_abs":"https://arxiv.org/abs/2105.09511v3","url_pdf":"https://arxiv.org/pdf/2105.09511v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"medical-image-segmentation-using-squeeze-and","repo_url":"https://github.com/askerlee/segtran","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"medical-image-segmentation-using-squeeze-and","repo_url":"https://github.com/maxiao0234/ttga","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"brain-tumor-segmentation","task_name":"Brain Tumor Segmentation"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"optic-cup-segmentation","task_name":"Optic Cup Segmentation"},{"task_slug":"optic-disc-segmentation","task_name":"Optic Disc Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"tumor-segmentation","task_name":"Tumor Segmentation"}],"methods":[{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"u-net","method_name":"U-Net"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/brain-tumor-segmentation-on-brats-2019","task":"Brain Tumor Segmentation","dataset":"BRATS 2019","model":"Segtran (i3d)","rank_in_archive_order":1,"of":5,"metrics":{"Avg.":"0.817","TC":"0.817"},"uses_additional_data":false},{"leaderboard":"/sota/brain-tumor-segmentation-on-brats-2019","task":"Brain Tumor Segmentation","dataset":"BRATS 2019","model":"Extension of nnU-Net","rank_in_archive_order":2,"of":5,"metrics":{"Avg.":"0.812","ET":"0.740","TC":"0.807","WT":"0.894"},"uses_additional_data":false},{"leaderboard":"/sota/brain-tumor-segmentation-on-brats-2019","task":"Brain Tumor Segmentation","dataset":"BRATS 2019","model":"Bag of tricks","rank_in_archive_order":3,"of":5,"metrics":{"ET":"0.729","TC":"0.802","WT":"0.895"},"uses_additional_data":false},{"leaderboard":"/sota/optic-cup-segmentation-on-refuge-challenge","task":"Optic Cup Segmentation","dataset":"REFUGE Challenge","model":"Segtran (EfficientNet-B4)","rank_in_archive_order":2,"of":2,"metrics":{"Dice":"0.872"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2105.09511","atlas_url":"https://app.syntology.ai/?focus=2105.09511","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.09511"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/maxiao0234/ttga","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/askerlee/segtran","reach":null}],"summary":{"ran":10,"ran_draft_wrong":2,"ran_honours":1,"unverified":7},"by_repo_kind":{"official":{"samples":20,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":20,"samples":[{"code_sha256_prefix":"8631456f20d9c1b9","entry":"LearnedSinuPosEmbedder","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8631456f20d9c1b9"}},{"code_sha256_prefix":"ce585f7154f9dc0c","entry":"LearnedSoftAggregate","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ce585f7154f9dc0c"}},{"code_sha256_prefix":"060455705ae47d70","entry":"MMPrivateMid","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"060455705ae47d70"}},{"code_sha256_prefix":"4af27a1b2e04f6f6","entry":"MMSharedMid","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4af27a1b2e04f6f6"}},{"code_sha256_prefix":"6f29f1ac1dd4e5ae","entry":"MultiHeadFeatTrans","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f29f1ac1dd4e5ae"}},{"code_sha256_prefix":"86e01c2857885263","entry":"NoneEmbedder","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86e01c2857885263"}},{"code_sha256_prefix":"4ad6430b01c09cb6","entry":"RandPosEmbedder","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4ad6430b01c09cb6"}},{"code_sha256_prefix":"f0592e287a093c48","entry":"SinuPosEmbedder","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0592e287a093c48"}},{"code_sha256_prefix":"9519a9bc56a4dc51","entry":"SlidingPosBiases2D","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9519a9bc56a4dc51"}},{"code_sha256_prefix":"3287ec7b8fd0c808","entry":"SlidingPosBiases3D","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3287ec7b8fd0c808"}},{"code_sha256_prefix":"3fa83a75e1d882d9","entry":"fracs_to_indices","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fa83a75e1d882d9"}},{"code_sha256_prefix":"46cc07a5b47f925b","entry":"multi_resize_shape","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"46cc07a5b47f925b"}},{"code_sha256_prefix":"8835d90bb9dd73b8","entry":"positionalencoding2d","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8835d90bb9dd73b8"}},{"code_sha256_prefix":"af5fb5367ebfbfb3","entry":"CrossAttFeatTrans","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"af5fb5367ebfbfb3"}},{"code_sha256_prefix":"5c226db1ced864bd","entry":"CrossMinceAttFeatTrans","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c226db1ced864bd"}},{"code_sha256_prefix":"06117d01318bffe8","entry":"ExpandedFeatTrans","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06117d01318bffe8"}},{"code_sha256_prefix":"acabdd901645b3e6","entry":"SegtranFusionEncoder","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"acabdd901645b3e6"}},{"code_sha256_prefix":"a5f46c66fa54514b","entry":"SegtranPosEncoder","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a5f46c66fa54514b"}},{"code_sha256_prefix":"efcfd592a205f3eb","entry":"SqueezedAttFeatTrans","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"efcfd592a205f3eb"}},{"code_sha256_prefix":"556f05e13b45cdee","entry":"resize_flat_features","repo":"askerlee/segtran","repo_kind":"official","path":"code/networks/segtran_shared.py","file_url":"https://github.com/askerlee/segtran/blob/HEAD/code/networks/segtran_shared.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"556f05e13b45cdee"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}