Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks

By Amy Jin, Serena Yeung, Jeffrey K. Jopling, Jonathan Krause, Dan Azagury, Arnold Milstein, Li Fei-Fei. IEEE Workshop/Winter Conference on Applications of Computer Vision, 2018

309 citations recorded by the source corpus.

Five billion people in the world lack access to quality surgical care. Surgeon skill varies dramatically, and many surgical patients suffer complications and avoidable harm. Improving surgical training and feedback would help to reduce the rate of complications—half of which have been shown to be preventable. To do this, it is essential to assess operative skill, a process that currently requires experts and is manual, time consuming, and subjective. In this work, we introduce an approach to automatically assess surgeon performance by tracking and analyzing tool movements in surgical videos, leveraging region-based convolutional neural networks. In order to study this problem, we also introduce a new dataset, m2cai16-tool-locations, which extends the m2cai16-tool dataset with spatial bounds of tools. While previous methods have addressed tool presence detection, ours is the first to not only detect presence but also spatially localize surgical tools in real-world laparoscopic surgical videos. We show that our method both effectively detects the spatial bounds of tools as well as significantly outperforms existing methods on tool presence detection. We further demonstrate the ability of our method to assess surgical quality through analysis of tool usage patterns, movement range, and economy of motion.

View Paper: https://www.semanticscholar.org/paper/d987bc0133f3331ef89ea1e50f5c57acfe17db9b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning-enabled medical computer vision (2021, background): “Some studies analyze tool movement or other cues to assess surgeon skill 119,121,123,124 , through established ratings such as the Global Operative Assessment of Laparoscopic Skills (GOALS) criteria for laparoscopic surgery 125 .”
A guide to deep learning in healthcare (2019, background): “Modern genomic technologies collect a wide variety of measurements, from an individual’s DNA sequence to the quantity of”
Surgical data science – from concepts toward clinical translation (2020, background): “m2cai16-tool-locations (Jin et al., 2018) cholecystectomy in-vivo, human video instrument bounding box 2016”