Outlier Processing in Multimodal Emotion Recognition

Abstract

Automatic emotion recognition plays a key role in human-computer interactions. Multimodal emotion recognition has attracted much attention in recent years. When multimodalities are used, different modalities interact with each other and the obtained results tend to be accurate in general. However, there are also cases of unimodal anomalies. Most of the existing studies do not take into account the existence of outliers in the multimodality, which leads to low accuracy of the prediction results. This paper proposes fuzzy weighted support vector machine for regression (FWSVR) to deal with outliers and prediction errors. We design an automatic affective recognition model structure to analyze continuous dimension emotions based on multimodality (audio and visual). The LIRIS-ACCEDE database is used in this work. Experimental results indicate that the concordance correlation coefficient (CCC) is 0.9456 for arousal and 0.9183 for valence on the test set. The fusion result obtained when using fuzzy weighting is much better than the direct fusion one.

Keywords

• Multimodal emotion recognition
• fuzzy weighted support vector machine for regression (FWSVR)
• continuous dimension emotional space
• outlier processing