Papers › Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets
Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets
Chinedu Innocent Nwoye, Cristians Gonzalez, Tong Yu, Pietro Mascagni, Didier Mutter, Jacques Marescaux, Nicolas Padoy
Recognition of surgical activity is an essential component to develop context-aware decision support for the operating room. In this work, we tackle the recognition of fine-grained activities, modeled as action triplets <instrument, verb, target> representing the tool activity. To this end, we introduce a new laparoscopic dataset, CholecT40, consisting of 40 videos from the public dataset Cholec80 in which all frames have been annotated using 128 triplet classes. Furthermore, we present an approach to recognize these triplets directly from the video data. It relies on a module called Class Activation Guide (CAG), which uses the instrument activation maps to guide the verb and target recognition. To model the recognition of multiple triplets in the same frame, we also propose a trainable 3D Interaction Space, which captures the associations between the triplet components. Finally, we demonstrate the significance of these contributions via several ablation studies and comparisons to baselines on CholecT40.
In Syntology View this paper on Syntology: its page in Syntology's graph, with its repositories and citations.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Datasets
Introduced by this paper, per the archive.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Triplet Recognition | CholecT40 | Tripnet | mAP | 18.95 | #1 of 1 | Archive leaderboard | report |
| Action Triplet Recognition | CholecT50 | Tripnet (TensorFlow v1) | Mean AP | 20.0 | #6 of 6 | Archive leaderboard | report |
| Action Triplet Recognition | CholecT50 (Challenge) | Tripnet (TensorFlow v1) | mAP | 23.4 | #20 of 27 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Introduced by this paper: CAG
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections