Decoding Bilingual EEG Signals With Complex Semantics Using Adaptive Graph Attention Convolutional Network

Abstract

Decoding neural signals of silent reading with Brain-Computer Interface (BCI) techniques presents a fast and intuitive communication method for severely aphasia patients. Electroencephalogram (EEG) acquisition is convenient and easily wearable with high temporal resolution. However, existing EEG-based decoding units primarily concentrate on individual words due to their low signal-to-noise ratio, rendering them insufficient for facilitating daily communication. Decoding at the word level is less efficient than decoding at the phrase or sentence level. Furthermore, with the popularity of multilingualism, decoding EEG signals with complex semantics under multiple languages is highly urgent and necessary. To the best of our knowledge, there is currently no research on decoding EEG signals during silent reading of complex semantics, let alone decoding silent reading EEG signals with complex semantics for bilingualism. Moreover, the feasibility of decoding such signals remains to be investigated. In this work, we collect silent reading EEG signals of 9 English Phrases (EP), 7 English Sentences (ES), 10 Chinese Phrases (CP), and 7 Chinese Sentences (CS) from the subject within 26 days. We propose a novel Adaptive Graph Attention Convolution Network (AGACN) for classification. Experimental results demonstrate that our proposed method outperforms state-of-the-art methods, achieving the highest classification accuracy of 54.70%, 62.26%, 44.55%, and 57.14% for silent reading EEG signals of EP, ES, CP, and CS, respectively. Moreover, our results prove the feasibility of complex semantics EEG signal decoding. This work will aid aphasic patients in achieving regular communication while providing novel ideas for neural signal decoding research.

Keywords

• Brain-computer interface
• EEG signals
• complex semantics
• silent reading
• multiple languages