Semantic Audiovisual Single-trial Detection Based on the New Generation of Magnetoencephalography

Abstract

In order to decode the difference between audiovisual bimodal and unimodal responses of the human brain in semantic context, this study designed a related task paradigm and applied a new generation magnetoencephalogram combined with the machine learning model to analyze the collected signals from three perspectives: behavioral response, event-related field (ERF) and single-trial detection. Results show that the unimodal semantic response was mainly concentrated in the occipital cortex, while the bimodal semantic response was mainly concentrated in the parietal cortex. At the same time, respondents' response rate and the detection accuracy of single-trial in bimodal mode were significantly higher than that in unimodal mode. Moreover, the support vector machine (SVM) showed the best classification performance among the four machine learning models, with an average classification accuracy of 75.16% for within-subject classification and 80.56% for between-subject classification. This research concludes that the combination of optically pumped magnetometer-magnetoencephalography (OPM-MEG) and machine learning model provides an efficient approach to decode the difference between audiovisual bimodal and unimodal responses of the human brain in semantic context.

Keywords

• opm-meg
• semantic
• audiovisual bimodal
• machine learning
• event-related field