{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cholectriplet2021-a-benchmark-challenge-for","title":"CholecTriplet2021: A benchmark challenge for surgical action triplet recognition","arxiv_id":"2204.04746","date":"2022-04-10","proceeding":null,"authors":["Chinedu Innocent Nwoye","Deepak Alapatt","Tong Yu","Armine Vardazaryan","Fangfang Xia","Zixuan Zhao","Tong Xia","Fucang Jia","Yuxuan Yang","Hao Wang","Derong Yu","Guoyan Zheng","Xiaotian Duan","Neil Getty","Ricardo Sanchez-Matilla","Maria Robu","Li Zhang","Huabin Chen","Jiacheng Wang","Liansheng Wang","Bokai Zhang","Beerend Gerats","Sista Raviteja","Rachana Sathish","Rong Tao","Satoshi Kondo","Winnie Pang","Hongliang Ren","Julian Ronald Abbing","Mohammad Hasan Sarhan","Sebastian Bodenstedt","Nithya Bhasker","Bruno Oliveira","Helena R. Torres","Li Ling","Finn Gaida","Tobias Czempiel","João L. Vilaça","Pedro Morais","Jaime Fonseca","Ruby Mae Egging","Inge Nicole Wijma","Chen Qian","GuiBin Bian","Zhen Li","Velmurugan Balasubramanian","Debdoot Sheet","Imanol Luengo","Yuanbo Zhu","Shuai Ding","Jakob-Anton Aschenbrenner","Nicolas Elini van der Kar","Mengya Xu","Mobarakol Islam","Lalithkumar Seenivasan","Alexander Jenke","Danail Stoyanov","Didier Mutter","Pietro Mascagni","Barbara Seeliger","Cristians Gonzalez","Nicolas Padoy"],"abstract":"Context-aware decision support in the operating room can foster surgical safety and efficiency by leveraging real-time feedback from surgical workflow analysis. Most existing works recognize surgical activities at a coarse-grained level, such as phases, steps or events, leaving out fine-grained interaction details about the surgical activity; yet those are needed for more helpful AI assistance in the operating room. Recognizing surgical actions as triplets of <instrument, verb, target> combination delivers comprehensive details about the activities taking place in surgical videos. This paper presents CholecTriplet2021: an endoscopic vision challenge organized at MICCAI 2021 for the recognition of surgical action triplets in laparoscopic videos. The challenge granted private access to the large-scale CholecT50 dataset, which is annotated with action triplet information. In this paper, we present the challenge setup and assessment of the state-of-the-art deep learning methods proposed by the participants during the challenge. A total of 4 baseline methods from the challenge organizers and 19 new deep learning algorithms by competing teams are presented to recognize surgical action triplets directly from surgical videos, achieving mean average precision (mAP) ranging from 4.2% to 38.1%. This study also analyzes the significance of the results obtained by the presented approaches, performs a thorough methodological comparison between them, in-depth result analysis, and proposes a novel ensemble method for enhanced recognition. Our analysis shows that surgical workflow analysis is not yet solved, and also highlights interesting directions for future research on fine-grained surgical activity recognition which is of utmost importance for the development of AI in surgery.","url_abs":"https://arxiv.org/abs/2204.04746v2","url_pdf":"https://arxiv.org/pdf/2204.04746v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cholectriplet2021-a-benchmark-challenge-for","repo_url":"https://github.com/CAMMA-public/cholectriplet2021","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"cholectriplet2021-a-benchmark-challenge-for","repo_url":"https://github.com/CAMMA-public/cholect45","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"cholectriplet2021-a-benchmark-challenge-for","repo_url":"https://github.com/CAMMA-public/cholect50","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"cholectriplet2021-a-benchmark-challenge-for","repo_url":"https://github.com/camma-public/attention-tripnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"cholectriplet2021-a-benchmark-challenge-for","repo_url":"https://github.com/camma-public/rendezvous","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"cholectriplet2021-a-benchmark-challenge-for","repo_url":"https://github.com/camma-public/tripnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"action-triplet-recognition","task_name":"Action Triplet Recognition"},{"task_slug":"activity-recognition","task_name":"Activity Recognition"},{"task_slug":null,"task_name":"Triplet"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Trequartista","rank_in_archive_order":1,"of":27,"metrics":{"mAP":"38.1"},"uses_additional_data":true},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team 2Ai","rank_in_archive_order":2,"of":27,"metrics":{"mAP":"36.9"},"uses_additional_data":true},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team SIAT CAMI","rank_in_archive_order":3,"of":27,"metrics":{"mAP":"35.8"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team HFUT-MedIA","rank_in_archive_order":5,"of":27,"metrics":{"mAP":"32.9"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Rendezvous (TensorFlow v1)","rank_in_archive_order":7,"of":27,"metrics":{"mAP":"32.7"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team CITI SJTU","rank_in_archive_order":8,"of":27,"metrics":{"mAP":"32.0"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team ANL","rank_in_archive_order":9,"of":27,"metrics":{"mAP":"31.9"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Digital Surgery","rank_in_archive_order":10,"of":27,"metrics":{"mAP":"31.7"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Casia Robotics","rank_in_archive_order":14,"of":27,"metrics":{"mAP":"26.7"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Lsgroup","rank_in_archive_order":15,"of":27,"metrics":{"mAP":"26.3"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team J&M","rank_in_archive_order":16,"of":27,"metrics":{"mAP":"25.6"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Attention Tripnet (TensorFlow v1)","rank_in_archive_order":17,"of":27,"metrics":{"mAP":"25.5"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Ceaiik","rank_in_archive_order":18,"of":27,"metrics":{"mAP":"25.2"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team SJTU-IMR","rank_in_archive_order":19,"of":27,"metrics":{"mAP":"24.8"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team SK","rank_in_archive_order":21,"of":27,"metrics":{"mAP":"18.4"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team MMLAB","rank_in_archive_order":22,"of":27,"metrics":{"mAP":"18.1"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Band of Broeders","rank_in_archive_order":23,"of":27,"metrics":{"mAP":"16.0"},"uses_additional_data":true},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team NCT-TSO","rank_in_archive_order":24,"of":27,"metrics":{"mAP":"10.4"},"uses_additional_data":true},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team HFUT-NUS","rank_in_archive_order":25,"of":27,"metrics":{"mAP":"9.8"},"uses_additional_data":false},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team CAMP","rank_in_archive_order":26,"of":27,"metrics":{"mAP":"9.3"},"uses_additional_data":true},{"leaderboard":"/sota/action-triplet-recognition-on-cholect50-1","task":"Action Triplet Recognition","dataset":"CholecT50 (Challenge)","model":"Team Med Recognizer","rank_in_archive_order":27,"of":27,"metrics":{"mAP":"4.2"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}