Predicting Surgical Phases using CNN-NARX Neural Network

Abstract

Online recognition of surgical phases is essential to develop systems able to effectively conceive the workflow and provide relevant information to surgical staff during surgical procedures. These systems, known as context-aware system (CAS), are designed to assist surgeons, improve scheduling efficiency of operating rooms (ORs) and surgical team and promote a comprehensive perception and awareness of the OR. State-of-the-art studies for recognizing surgical phases have made use of data from different sources such as videos or binary usage signals from surgical tools. In this work, we propose a deep learning pipeline, namely a convolutional neural network (CNN) and a nonlinear autoregressive network with exogenous inputs (NARX), designed to predict surgical phases from laparoscopic videos. A convolutional neural network (CNN) is used to perform the tool classification task by automatically learning visual features from laparoscopic videos. The output of the CNN, which represents binary usage signals of surgical tools, is provided to a NARX neural network that performs a multistep-ahead predictions of surgical phases. Surgical phase prediction performance of the proposed pipeline was evaluated on a dataset of 80 cholecystectomy videos (Cholec80 dataset). Results show that the NARX model provides a good modelling of the temporal dependencies between surgical phases. However, more input signals are needed to improve the recognition accuracy.

Keywords

• surgical phase recognition
• convolutional neural network (cnn)
• narx neural network
• surgical process modelling
• cholecystectomy