A Cutting-Edge Approach to Decision-Making in Awake Neurosurgery Based on Hand Motion Recognition

Abstract

In Awake Neurosurgery (AN), the assessment of the patient’s capabilities is of paramount importance to minimize the risk of post-operative cognitive, language, and motor deficits. To retain fine hand motion capabilities, in the current clinical practice, a neuropsychologist assigns hand motion tasks to the patient during AN, and these tasks are evaluated by visual observation. This study introduces an innovative and non-invasive task evaluation method that employs an infrared stereo camera for precise hand pose acquisition. The primary contribution of this work is to offer objective, quantitative and reproducible task evaluations, addressing a critical aspect of patient care during AN procedures. The proposed method focuses on dynamic hand gestures recognition and utilizes unidirectional and bidirectional Long Short-Term Memory networks to assess common motor tasks during AN. To evaluate the tasks, we employ state-of-the-art features, both distance-based and angle-based, formed by the finger bones of the human hands. These feature vectors demonstrate a promising accuracy and inter-patient portability for the tasks under consideration, with mean accuracy exceeding 80% when tested on new patients, separate from those used in training. This approach provides an efficient solution for the identification and assessment of tasks, also eliminating the need for task-specific labeling. This, in turn, enhances usability and reduces the potential for human error. The proposed method has the potential to improve clinical decision-making in AN and offers reliable classification tools for neuropsychologists.

Keywords

• Awake neurosurgery
• decision making support
• hand gesture recognition
• leap motion controller
• long short term memory
• recurrent neural network