Real-time surgical instrument tracking in robot-assisted surgery using multi-domain convolutional neural network

Abstract

Image-based surgical instrument tracking in robot-assisted surgery is an active and challenging research area. Having a real-time knowledge of surgical instrument location is an essential part of a computer-assisted intervention system. Tracking can be used as visual feedback for servo control of a surgical robot or transformed as haptic feedback for surgeon–robot interaction. In this Letter, the authors apply a multi-domain convolutional neural network for fast 2D surgical instrument tracking considering the application for multiple surgical tools and use a focal loss to decrease the effect of easy negative examples. They further introduce a new dataset based on m2cai16-tool and their cadaver experiments due to the lack of established public surgical tool tracking dataset despite significant progress in this field. Their method is evaluated on the introduced dataset and outperforms the state-of-the-art real-time trackers.

Keywords

• neural nets
• medical robotics
• surgery
• time surgical instrument tracking
• robot-assisted surgery
• multidomain convolutional neural network
• active research area
• challenging research area
• real-time knowledge
• surgical instrument location
• computer-assisted intervention system
• surgical robot
• surgeon–robot interaction
• multiple surgical tools
• established public surgical tool tracking dataset