Validation of a convolutional neural network that reliably identifies electromyographic compound motor action potentials following train-of-four stimulation: an algorithm development experimental study

Abstract

Background: International guidelines recommend quantitative neuromuscular monitoring when administering neuromuscular blocking agents. The train-of-four count is important for determining the depth of block and appropriate reversal agents and doses. However, identifying valid compound motor action potentials (cMAPs) during surgery can be challenging because of low-amplitude signals and an inability to observe motor responses. A convolutional neural network (CNN) to classify cMAPs as valid or not might improve the accuracy of such determinations. Methods: We modified a high-accuracy CNN originally developed to identify handwritten numbers. For training, we used digitised electromyograph waveforms (TetraGraph) from a previous study of 29 patients and tuned the model parameters using leave-one-out cross-validation. External validation used a dataset of 19 patients from another study with the same neuromuscular block monitor but with different patient, surgical, and protocol characteristics. All patients underwent ulnar nerve stimulation at the wrist and the surface electromyogram was recorded from the adductor pollicis muscle. Results: The tuned CNN performed highly on the validation dataset, with an accuracy of 0.9997 (99% confidence interval 0.9994–0.9999) and F1 score=0.9998. Performance was equally good for classifying the four individual responses in the train-of-four sequence. The calibration plot showed excellent agreement between the predicted probabilities and the actual prevalence of valid cMAPs. Ten-fold cross-validation using all data showed similar high performance. Conclusions: The CNN distinguished valid cMAPs from artifacts after ulnar nerve stimulation at the wrist with >99.5% accuracy. Incorporation of such a process within quantitative electromyographic neuromuscular block monitors is feasible.

Keywords

• electromyography
• machine learning
• neural network
• neuromuscular block
• train-of-four