{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vivim-a-video-vision-mamba-for-medical-video","title":"Vivim: a Video Vision Mamba for Medical Video Segmentation","arxiv_id":"2401.14168","date":"2024-01-25","proceeding":null,"authors":["Yijun Yang","Zhaohu Xing","Chunwang Huang","Lei Zhu"],"abstract":"Medical video segmentation gains increasing attention in clinical practice due to the redundant dynamic references in video frames. However, traditional convolutional neural networks have a limited receptive field and transformer-based networks are mediocre in constructing long-term dependency from the perspective of computational complexity. This bottleneck poses a significant challenge when processing longer sequences in medical video analysis tasks using available devices with limited memory. Recently, state space models (SSMs), famous by Mamba, have exhibited impressive achievements in efficient long sequence modeling, which develops deep neural networks by expanding the receptive field on many vision tasks significantly. Unfortunately, vanilla SSMs failed to simultaneously capture causal temporal cues and preserve non-casual spatial information. To this end, this paper presents a Video Vision Mamba-based framework, dubbed as Vivim, for medical video segmentation tasks. Our Vivim can effectively compress the long-term spatiotemporal representation into sequences at varying scales with our designed Temporal Mamba Block. We also introduce an improved boundary-aware affine constraint across frames to enhance the discriminative ability of Vivim on ambiguous lesions. Extensive experiments on thyroid segmentation, breast lesion segmentation in ultrasound videos, and polyp segmentation in colonoscopy videos demonstrate the effectiveness and efficiency of our Vivim, superior to existing methods. The code is available at: https://github.com/scott-yjyang/Vivim. The dataset will be released once accepted.","url_abs":"https://arxiv.org/abs/2401.14168v4","url_pdf":"https://arxiv.org/pdf/2401.14168v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vivim-a-video-vision-mamba-for-medical-video","repo_url":"https://github.com/scott-yjyang/vivim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"lesion-segmentation","task_name":"Lesion Segmentation"},{"task_slug":"mamba","task_name":"Mamba"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"state-space-models","task_name":"State Space Models"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2401.14168","atlas_url":"https://app.syntology.ai/?focus=2401.14168","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.14168"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scott-yjyang/vivim","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":6,"ran_fixture":2,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":13,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"c6e70d5ac2de73bf","entry":"binary_entropy_loss","repo":"scott-yjyang/vivim","repo_kind":"official","path":"loss.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c6e70d5ac2de73bf"}},{"code_sha256_prefix":"0595d8b8521fd9ca","entry":"causal_conv1d_ref","repo":"scott-yjyang/vivim","repo_kind":"official","path":"causal-conv1d/causal_conv1d/causal_conv1d_interface.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/causal-conv1d/causal_conv1d/causal_conv1d_interface.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0595d8b8521fd9ca"}},{"code_sha256_prefix":"41839b27773cccf7","entry":"cv_random_flip","repo":"scott-yjyang/vivim","repo_kind":"official","path":"data_polyp.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/data_polyp.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"41839b27773cccf7"}},{"code_sha256_prefix":"00364c3c6b4b23e8","entry":"jaccard","repo":"scott-yjyang/vivim","repo_kind":"official","path":"misc2.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/misc2.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"00364c3c6b4b23e8"}},{"code_sha256_prefix":"418a04607e943f31","entry":"load_model_from_dict","repo":"scott-yjyang/vivim","repo_kind":"official","path":"modeling/utils.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/modeling/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"418a04607e943f31"}},{"code_sha256_prefix":"53bd64a4a61b8042","entry":"mask_iou","repo":"scott-yjyang/vivim","repo_kind":"official","path":"loss.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"53bd64a4a61b8042"}},{"code_sha256_prefix":"b8fc70f2a16b23cc","entry":"randomCrop","repo":"scott-yjyang/vivim","repo_kind":"official","path":"data_polyp.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/data_polyp.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8fc70f2a16b23cc"}},{"code_sha256_prefix":"8d40bcc4f2cf3edb","entry":"randomRotation","repo":"scott-yjyang/vivim","repo_kind":"official","path":"data_polyp.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/data_polyp.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d40bcc4f2cf3edb"}},{"code_sha256_prefix":"797eafbb72dabcfd","entry":"structure_loss","repo":"scott-yjyang/vivim","repo_kind":"official","path":"train_pl_polyp.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/train_pl_polyp.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"797eafbb72dabcfd"}},{"code_sha256_prefix":"a0c651abef32e608","entry":"structure_loss","repo":"scott-yjyang/vivim","repo_kind":"official","path":"modeling/utils.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/modeling/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a0c651abef32e608"}},{"code_sha256_prefix":"e4f26582292cf067","entry":"cross_entropy_loss","repo":"scott-yjyang/vivim","repo_kind":"official","path":"loss.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e4f26582292cf067"}},{"code_sha256_prefix":"1b2b6e788bbbb51a","entry":"dice","repo":"scott-yjyang/vivim","repo_kind":"official","path":"misc2.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/misc2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1b2b6e788bbbb51a"}},{"code_sha256_prefix":"a0be218082173339","entry":"precision","repo":"scott-yjyang/vivim","repo_kind":"official","path":"misc2.py","file_url":"https://github.com/scott-yjyang/vivim/blob/HEAD/misc2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a0be218082173339"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}