Papers › Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification

Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification

21 Jun 2018arXiv:1806.08089archive 2025-07-28

Daochang Liu, Tingting Jiang

Recognition of surgical gesture is crucial for surgical skill assessment and efficient surgery training. Prior works on this task are based on either variant graphical models such as HMMs and CRFs, or deep learning models such as Recurrent Neural Networks and Temporal Convolutional Networks. Most of the current approaches usually suffer from over-segmentation and therefore low segment-level edit scores. In contrast, we present an essentially different methodology by modeling the task as a sequential decision-making process. An intelligent agent is trained using reinforcement learning with hierarchical features from a deep model. Temporal consistency is integrated into our action design and reward mechanism to reduce over-segmentation errors. Experiments on JIGSAWS dataset demonstrate that the proposed method performs better than state-of-the-art methods in terms of the edit score and on par in frame-wise accuracy. Our code will be released later.

Paper PDF Code

Code

Finspire13/RL-Surgical-Gesture-Segmentation mentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action Segmentation Classification Decision Making Deep Reinforcement Learning General Classification Reinforcement Learning Reinforcement Learning (RL)Segmentation Sequential Decision Making Surgical Gesture Recognition reinforcement-learning

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Action Segmentation	JIGSAWS	RL (full)	Accuracy	81.43	#3 of 7	Archive leaderboard	report
Action Segmentation	JIGSAWS	RL (full)	Edit Distance	87.96	#3 of 7	Archive leaderboard	report
Action Segmentation	JIGSAWS	RL (full)	F1@10	92.0	#3 of 7	Archive leaderboard	report
Action Segmentation	JIGSAWS	RL (full)	F1@25	90.5	#3 of 7	Archive leaderboard	report
Action Segmentation	JIGSAWS	RL (full)	F1@50	82.2	#3 of 7	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections