{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-prompting-polyp-segmentation-in","title":"Self-Prompting Polyp Segmentation in Colonoscopy using Hybrid Yolo-SAM 2 Model","arxiv_id":"2409.09484","date":"2024-09-14","proceeding":null,"authors":["Mobina Mansoori","Sajjad Shahabodini","Jamshid Abouei","Konstantinos N. Plataniotis","Arash Mohammadi"],"abstract":"Early diagnosis and treatment of polyps during colonoscopy are essential for reducing the incidence and mortality of Colorectal Cancer (CRC). However, the variability in polyp characteristics and the presence of artifacts in colonoscopy images and videos pose significant challenges for accurate and efficient polyp detection and segmentation. This paper presents a novel approach to polyp segmentation by integrating the Segment Anything Model (SAM 2) with the YOLOv8 model. Our method leverages YOLOv8's bounding box predictions to autonomously generate input prompts for SAM 2, thereby reducing the need for manual annotations. We conducted exhaustive tests on five benchmark colonoscopy image datasets and two colonoscopy video datasets, demonstrating that our method exceeds state-of-the-art models in both image and video segmentation tasks. Notably, our approach achieves high segmentation accuracy using only bounding box annotations, significantly reducing annotation time and effort. This advancement holds promise for enhancing the efficiency and scalability of polyp detection in clinical settings https://github.com/sajjad-sh33/YOLO_SAM2.","url_abs":"https://arxiv.org/abs/2409.09484v1","url_pdf":"https://arxiv.org/pdf/2409.09484v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-prompting-polyp-segmentation-in","repo_url":"https://github.com/sajjad-sh33/yolo_sam2","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"medical-image-segmentation","task_name":"Medical Image Segmentation"},{"task_slug":"polyp-segmentation","task_name":"Polyp Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"video-polyp-segmentation","task_name":"Video Polyp Segmentation"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"sam","method_name":"SAM"},{"method_slug":"yolov8","method_name":"YOLOv8"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/medical-image-segmentation-on-cvc-clinicdb","task":"Medical Image Segmentation","dataset":"CVC-ClinicDB","model":"Yolo-SAM 2","rank_in_archive_order":5,"of":48,"metrics":{"mIoU":"0.909","mean Dice":"0.951"},"uses_additional_data":false},{"leaderboard":"/sota/medical-image-segmentation-on-kvasir-seg","task":"Medical Image Segmentation","dataset":"Kvasir-SEG","model":"Yolo-SAM 2","rank_in_archive_order":49,"of":58,"metrics":{"mIoU":"0.764","mean Dice":"0.866"},"uses_additional_data":false},{"leaderboard":"/sota/polyp-segmentation-on-polypgen","task":"Polyp Segmentation","dataset":"PolypGen","model":"YOlO-SAM 2","rank_in_archive_order":1,"of":2,"metrics":{"Dice":"0.808","Precision":"0.858","Recall":"0.764","mIoU":"0.678"},"uses_additional_data":false},{"leaderboard":"/sota/video-polyp-segmentation-on-sun-seg-easy","task":"Video Polyp Segmentation","dataset":"SUN-SEG-Easy (Unseen)","model":"YOLO-SAM 2","rank_in_archive_order":1,"of":18,"metrics":{"Dice":"0.90","S measure":"0.9","Sensitivity":"83.7","mean E-measure":"93.8","mean F-measure":"93.8"},"uses_additional_data":false},{"leaderboard":"/sota/video-polyp-segmentation-on-sun-seg-hard","task":"Video Polyp Segmentation","dataset":"SUN-SEG-Hard (Unseen)","model":"YOLO-SAM 2","rank_in_archive_order":1,"of":18,"metrics":{"Dice":"0.902","S-Measure":"0.894","Sensitivity":"0.852","mean E-measure":"0.941","mean F-measure":"0.932"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}