Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows

Abstract

We introduce a wearable-based recognition system for the classification of natural hand gestures during dynamic activities with surgical instruments. An armbandbased circular setup of eight EMG-sensors was used to superficially measure the muscle activation signals over the broadest cross-section of the lower arm. Instrument-specific surface EMG (sEMG) data acquisition was performed for 5 distinct instruments. In a first proof-of-concept study, EMG data were analyzed for unique signal courses and features, and in a subsequent classification, both decision tree (DTR) and shallow artificial neural network (ANN) classifiers were trained. For DTR, an ensemble bagging approach reached precision and recall rates of 0.847 and 0.854, respectively. The ANN network architecture was configured to mimic the ensemble-like structure of the DTR and achieved 0.952 and 0.953 precision and recall rates, respectively. In a subsequent multi-user study, classification achieved 70 % precision. Main errors potentially arise for instruments with similar gripping style and performed actions, interindividual variations in the acquisition procedure as well as muscle tone and activation magnitude. Compared to hand-mounted sensor systems, the lower arm setup does not alter the haptic experience or the instrument gripping, which is critical, especially in an intraoperative environment. Currently, drawbacks of the fixed consumer product setup are the limited data sampling rate and the denial of frequency features into the processing pipeline.

Keywords

• natural user interface
• gesture recognition
• workflow recognition
• emg processing
• machine learning