{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/weakly-supervised-convolutional-lstm-approach","title":"Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos","arxiv_id":"1812.01366","date":"2018-12-04","proceeding":null,"authors":["Chinedu Innocent Nwoye","Didier Mutter","Jacques Marescaux","Nicolas Padoy"],"abstract":"Purpose: Real-time surgical tool tracking is a core component of the future\nintelligent operating room (OR), because it is highly instrumental to analyze\nand understand the surgical activities. Current methods for surgical tool\ntracking in videos need to be trained on data in which the spatial positions of\nthe tools are manually annotated. Generating such training data is difficult\nand time-consuming. Instead, we propose to use solely binary presence\nannotations to train a tool tracker for laparoscopic videos. Methods: The\nproposed approach is composed of a CNN + Convolutional LSTM (ConvLSTM) neural\nnetwork trained end-to-end, but weakly supervised on tool binary presence\nlabels only. We use the ConvLSTM to model the temporal dependencies in the\nmotion of the surgical tools and leverage its spatio-temporal ability to smooth\nthe class peak activations in the localization heat maps (Lh-maps).\n  Results: We build a baseline tracker on top of the CNN model and demonstrate\nthat our approach based on the ConvLSTM outperforms the baseline in tool\npresence detection, spatial localization, and motion tracking by over 5.0%,\n13.9%, and 12.6%, respectively.\n  Conclusions: In this paper, we demonstrate that binary presence labels are\nsufficient for training a deep learning tracking model using our proposed\nmethod. We also show that the ConvLSTM can leverage the spatio-temporal\ncoherence of consecutive image frames across a surgical video to improve tool\npresence detection, spatial localization, and motion tracking.\n  keywords: Surgical workflow analysis, tool tracking, weak supervision,\nspatio-temporal coherence, ConvLSTM, endoscopic videos","url_abs":"http://arxiv.org/abs/1812.01366v2","url_pdf":"http://arxiv.org/pdf/1812.01366v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"weakly-supervised-convolutional-lstm-approach","repo_url":"https://github.com/CAMMA-public/ConvLSTM-Surgical-Tool-Tracker","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"instrument-recognition","task_name":"Instrument Recognition"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"surgical-tool-detection","task_name":"Surgical tool detection"},{"task_slug":"video-object-tracking","task_name":"Video Object Tracking"},{"task_slug":"weakly-supervised-object-localization","task_name":"Weakly-Supervised Object Localization"}],"methods":[{"method_slug":"1d-cnn","method_name":"1D CNN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/surgical-tool-detection-on-cholec80","task":"Surgical tool detection","dataset":"Cholec80","model":"ConvLSTM tracker","rank_in_archive_order":2,"of":6,"metrics":{"mAP":"92.9"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}