A multimodal fusion network based on a cross-attention mechanism for the classification of Parkinsonian tremor and essential tremor

Abstract

Abstract Parkinsonian tremor (PT) and Essential tremor (ET) exist as upper limb tremors in clinical practice. Notably, their types of trembling share similar presentations and overlapping frequencies. To enhance objectivity and efficiency in the diagnosis of these two diseases, there is a pressing need for more objective tremor classification procedures. This study proposes a novel multimodal fusion network based on a cross-attention mechanism (MFCA-Net) to automate the classification of upper limb tremors between PTs and ETs. To this end, 140 patients with PTs and ETs were recruited, and acceleration and surface electromyography (sEMG) signals were collected from the forearm during tremor episodes. To comprehensively capture the global and local features of input signals, a multiscale convolution in MFCA-Net was designed. Furthermore, the cross-attention mechanism was applied to fuse the features of the two input signals. The results demonstrate that the final classification accuracy exceeded 97.18% when MFCA-Net was used. Compared with the single acceleration signal and single sEMG signal inputs, the recognition accuracies increased by 18.91% and 10.04%, respectively. Therefore, the proposed MFCA-Net in this study serves as an objective and potential tool for assisting clinicians in the diagnosis of PT and ET patients.

Keywords

• Parkinsonian tremor
• Essential tremor
• Deep learning
• Multimodal fusion
• Cross-attention
• Tremor classification