An Empirical Analysis of Multimodal Affective Computing Approaches for Advancing Emotional Intelligence in Artificial Intelligence for Healthcare

Abstract

Due to its potential use in evaluating mental health and enhancing patient care, emotion recognition using physiological signals, such as EEG, ECG, and EMG, has drawn a lot of attention in the healthcare industry. However, better neural network architecture and training parameter optimization is needed to achieve accurate and dependable emotion classification. This research aims to investigate the impact of various configurations of neural network architectures, proposed multimodal fusion network and training parameters on the classification performance of emotion recognition systems utilizing EEG, ECG, and EMG signals. Nine experiments were conducted, each representing a unique combination of hyperparameters, including EEG filters, kernel sizes, pooling sizes, ECG filters, EMG filters, LSTM units, dropout rates, epochs, and batch sizes. The experiments were designed to systematically explore the effects of these hyperparameters on classification accuracy, robustness, and computational efficiency. The experiments revealed that the choice of hyperparameters significantly influences the performance of emotion recognition systems. Proposed model achieved higher average accuracies, with Experiment 3 exhibiting the highest accuracy of 87.83%. However, the model also incurred higher computational costs in terms of training time and memory usage. Additionally, experiments with higher dropout rates demonstrated a slight decrease in accuracy compared to those with lower dropout rates, emphasizing the importance of balancing regularization and model complexity. The findings underscore the importance of fine-tuning hyperparameters and selecting appropriate neural network architectures to optimize the performance of emotion recognition systems. Advanced deep learning techniques and physiological signal processing, healthcare providers can develop robust and accurate emotion recognition systems to enhance patient care and support personalized interventions. Further research may focus on exploring novel architectures and refining training strategies to continue improving the efficacy and efficiency of AI-driven emotional intelligence solutions in healthcare delivery.

Keywords

• Emotion recognition
• EEG
• ECG
• EMG
• multi-modal fusion
• neural network