Decoding Gestures in Electromyography: Spatiotemporal Graph Neural Networks for Generalizable and Interpretable Classification

Abstract

In recent years, significant strides in deep learning have propelled the advancement of electromyography (EMG)-based upper-limb gesture recognition systems, yielding notable successes across a spectrum of domains, including rehabilitation, orthopedics, robotics, and human-computer interaction. Despite these achievements, prevailing methodologies often overlook the intrinsic physical configurations and interconnectivity of multi-channel sensory inputs, resulting in a failure to adequately capture relational information embedded within the connections of deployed EMG sensor network topology. This oversight poses a significant challenge, impeding the extraction of crucial features from collaborative multi-channel EMG inputs and subsequently constraining model performance, generalizability, and interpretability. To address these limitations, we introduce novel graph structures meticulously crafted to encapsulate the spatial proximity of distributed EMG sensors and the temporal adjacency of EMG signals. Harnessing these tailored graph structures, we present Graph Convolution Network (GCN)-based classification models adept at effectively extracting and aggregating key features associated with various gestures. Our methodology exhibits remarkable efficacy, achieving state-of-the-art performance across five publicly available datasets, thus underscoring its prowess in gesture recognition tasks. Furthermore, our approach provides interpretable insights into muscular activation patterns, thereby reaffirming the practical effectiveness of our GCN model. Moreover, we show the effectiveness of our graph-based input structure and GCN-based classifier in maintaining high accuracy even with reduced sensor configurations, suggesting their potential for seamless integration into AI-powered rehabilitation strategies utilizing EMG-based gesture classification systems.

Keywords

• Explainable AI
• generalizability
• graph neural network
• graph representation
• hand gesture classification
• surface electromyography