The Process of Comparing Images of Emotional Expressions

Abstract

Introduction. We conducted an experiment in the paradigm of direct comparison of images of strong and weakly expressed emotional expressions with a detailed justification of the assessment made and registration of eye movements. Methods. Photo images from the VEPEL database (video images of natural transient facial expressions: joy, sadness, fear, surprise, anger, disgust, calm face) were used as stimulus material. The subjects were students of Moscow universities (72 people, of which 10 men, 62 women; age from 18 to 39 years, average age = 22.0, standard deviation = 4.0. Exposure time is unlimited, until the justification is completed. Research objective: image comparison (rank scale of similarity between images from 1 to 9) with registration of eye movements. Results. Based on individual assessments of similarity between images of emotional expressions, the reconstruction of the two-dimensional space was performed using the multidimensional scaling method. The reconstruction is described by Core Affect model by J. Russell. The presence of individual variability of similarity scores (the tendency to select a certain range of scores) is shown. The following individual indicators were singled out for further search of possible predictors: the average similarity score between images, the standard deviation of the similarity score between images, and the average individual duration of fixations. The presence of variability of estimates for different pairs of compared images is shown. The minimum variability of similarity estimates is achieved for the next pairs: fear–fear weak, joy – joy weak; anger – anger weak; disgust weak – anger; neutral – sadness weak. The maximum variability of similarity estimates is achieved for pairs of joy weak – fear weak; joy – fear weak; sadness – joy weak; joy weak – anger weak; neutral – joy weak. The analysis of the duration of visual fixations during the similarity assessment was carried out. It is shown that different similarity scores correspond to different distribution patterns of fixation durations in the evaluation process. Discussion. Based on our results, we can conclude that there are several convergent evaluation justification processes based on an initial similarity score between images.

Keywords

• emotional expressions
• emotions
• comparison
• core affect
• valency
• scoring process
• eye movements
• fixation
• decision time
• multivariate scaling