{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dissecting-self-supervised-learning-methods","title":"Dissecting Self-Supervised Learning Methods for Surgical Computer Vision","arxiv_id":"2207.00449","date":"2022-07-01","proceeding":null,"authors":["Sanat Ramesh","Vinkle Srivastav","Deepak Alapatt","Tong Yu","Aditya Murali","Luca Sestini","Chinedu Innocent Nwoye","Idris Hamoud","Saurav Sharma","Antoine Fleurentin","Georgios Exarchakis","Alexandros Karargyris","Nicolas Padoy"],"abstract":"The field of surgical computer vision has undergone considerable breakthroughs in recent years with the rising popularity of deep neural network-based methods. However, standard fully-supervised approaches for training such models require vast amounts of annotated data, imposing a prohibitively high cost; especially in the clinical domain. Self-Supervised Learning (SSL) methods, which have begun to gain traction in the general computer vision community, represent a potential solution to these annotation costs, allowing to learn useful representations from only unlabeled data. Still, the effectiveness of SSL methods in more complex and impactful domains, such as medicine and surgery, remains limited and unexplored. In this work, we address this critical need by investigating four state-of-the-art SSL methods (MoCo v2, SimCLR, DINO, SwAV) in the context of surgical computer vision. We present an extensive analysis of the performance of these methods on the Cholec80 dataset for two fundamental and popular tasks in surgical context understanding, phase recognition and tool presence detection. We examine their parameterization, then their behavior with respect to training data quantities in semi-supervised settings. Correct transfer of these methods to surgery, as described and conducted in this work, leads to substantial performance gains over generic uses of SSL - up to 7.4% on phase recognition and 20% on tool presence detection - as well as state-of-the-art semi-supervised phase recognition approaches by up to 14%. Further results obtained on a highly diverse selection of surgical datasets exhibit strong generalization properties. The code is available at https://github.com/CAMMA-public/SelfSupSurg.","url_abs":"https://arxiv.org/abs/2207.00449v3","url_pdf":"https://arxiv.org/pdf/2207.00449v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dissecting-self-supervised-learning-methods","repo_url":"https://github.com/camma-public/selfsupsurg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"action-triplet-recognition","task_name":"Action Triplet Recognition"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"surgical-phase-recognition","task_name":"Surgical phase recognition"},{"task_slug":"surgical-tool-detection","task_name":"Surgical tool detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"colorjitter","method_name":"ColorJitter"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"nt-xent","method_name":"NT-Xent"},{"method_slug":"random-gaussian-blur","method_name":"Random Gaussian Blur"},{"method_slug":"random-resized-crop","method_name":"Random Resized Crop"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"simclr","method_name":"SimCLR"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"MoCo V2 Surg SSL - Rendezvous head","rank_in_archive_order":4,"of":27,"metrics":{"mAP":"35.7"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-endoscapes","task":"Semantic Segmentation","dataset":"Endoscapes","model":"MoCo V2 Surg SSL -  DeepLabv3+ head","rank_in_archive_order":1,"of":2,"metrics":{"Mean F1":"73.2"},"uses_additional_data":false},{"leaderboard":"/sota/surgical-phase-recognition-on-cholec80-1","task":"Surgical phase recognition","dataset":"Cholec80","model":"MoCo V2 Surg SSL - TCN head","rank_in_archive_order":4,"of":6,"metrics":{"F1":"81.6"},"uses_additional_data":false},{"leaderboard":"/sota/surgical-phase-recognition-on-heichole","task":"Surgical phase recognition","dataset":"HeiChole Benchmark","model":"MoCo V2 Surg SSL - TCN head","rank_in_archive_order":5,"of":5,"metrics":{"F1":"64.7"},"uses_additional_data":false},{"leaderboard":"/sota/surgical-tool-detection-on-cholec80","task":"Surgical tool detection","dataset":"Cholec80","model":"MoCo V2 Surg SSL - FCN head","rank_in_archive_order":1,"of":6,"metrics":{"mAP":"93.5"},"uses_additional_data":false},{"leaderboard":"/sota/surgical-tool-detection-on-heichole-benchmark","task":"Surgical tool detection","dataset":"HeiChole Benchmark","model":"MoCo V2 Surg SSL - FCN head","rank_in_archive_order":1,"of":1,"metrics":{"mAP":"66.9"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}