{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lvit-language-meets-vision-transformer-in","title":"LViT: Language meets Vision Transformer in Medical Image Segmentation","arxiv_id":"2206.14718","date":"2022-06-29","proceeding":null,"authors":["Zihan Li","Yunxiang Li","Qingde Li","Puyang Wang","Dazhou Guo","Le Lu","Dakai Jin","You Zhang","Qingqi Hong"],"abstract":"Deep learning has been widely used in medical image segmentation and other aspects. However, the performance of existing medical image segmentation models has been limited by the challenge of obtaining sufficient high-quality labeled data due to the prohibitive data annotation cost. To alleviate this limitation, we propose a new text-augmented medical image segmentation model LViT (Language meets Vision Transformer). In our LViT model, medical text annotation is incorporated to compensate for the quality deficiency in image data. In addition, the text information can guide to generate pseudo labels of improved quality in the semi-supervised learning. We also propose an Exponential Pseudo label Iteration mechanism (EPI) to help the Pixel-Level Attention Module (PLAM) preserve local image features in semi-supervised LViT setting. In our model, LV (Language-Vision) loss is designed to supervise the training of unlabeled images using text information directly. For evaluation, we construct three multimodal medical segmentation datasets (image + text) containing X-rays and CT images. Experimental results show that our proposed LViT has superior segmentation performance in both fully-supervised and semi-supervised setting. The code and datasets are available at https://github.com/HUANGLIZI/LViT.","url_abs":"https://arxiv.org/abs/2206.14718v4","url_pdf":"https://arxiv.org/pdf/2206.14718v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lvit-language-meets-vision-transformer-in","repo_url":"https://github.com/huanglizi/lvit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"pseudo-label","task_name":"Pseudo Label"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"text-annotation","task_name":"text annotation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"channel-wise-cross-attention","method_name":"Channel-wise Cross Attention"},{"method_slug":"channel-wise-cross-fusion-transformer","method_name":"Channel-wise Cross Fusion Transformer"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"instance-normalization","method_name":"Instance Normalization"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"uctransnet","method_name":"UCTransNet"},{"method_slug":"unet","method_name":"UNet++"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/medical-image-segmentation-on-monuseg","task":"Medical Image Segmentation","dataset":"MoNuSeg","model":"LViT-L","rank_in_archive_order":4,"of":15,"metrics":{"F1":"81.01","IoU":"68.2"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-monuseg","task":"Medical Image Segmentation","dataset":"MoNuSeg","model":"LViT-LW","rank_in_archive_order":6,"of":15,"metrics":{"F1":"80.66","IoU":"67.71"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-monuseg","task":"Medical Image Segmentation","dataset":"MoNuSeg","model":"UCTransNet","rank_in_archive_order":7,"of":15,"metrics":{"F1":"79.87","IoU":"66.68"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-monuseg","task":"Medical Image Segmentation","dataset":"MoNuSeg","model":"GTUNet","rank_in_archive_order":10,"of":15,"metrics":{"F1":"79.26","IoU":"65.94"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-monuseg","task":"Medical Image Segmentation","dataset":"MoNuSeg","model":"UNet++","rank_in_archive_order":11,"of":15,"metrics":{"F1":"77.01","IoU":"63.04"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2206.14718","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.14718"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huanglizi/lvit","reach":null}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"43ef84aa490c7681","entry":"logger_config","repo":"huanglizi/lvit","repo_kind":"official","path":"train_model.py","file_url":"https://github.com/huanglizi/lvit/blob/HEAD/train_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43ef84aa490c7681"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}