Effects of Facial Expressions and Gestures on the Trustworthiness of a Person

Abstract

Trust is a fundamental element in human relationships, playing a crucial role in decision-making processes. Despite its significance, numerous dimensions of perceived trustworthiness remain unexplored and warrant further investigation. Previous literature has highlighted the influence of emotions and sentiments on how individuals perceive trustworthiness, with visual, vocal, and behavioral cues serving as essential markers. This preliminary study aims to expand the existing knowledge in the field by investigating trustworthiness traits manifested through facial and gesture expressions in emotional videos across diverse cultural contexts. To address this objective, an annotation platform was developed to collect annotation data using the benchmarked One-Minute-Gradual Emotion (OMG) audiovisual corpus, enabling the annotation of actors’ perceived trustworthiness levels alongside other inquiries related to emotional state, gesture, activeness, comfort, and speech integrity. The findings of this study demonstrate a positive correlation between higher levels of speaker activity, faster gesturing, and a relaxed demeanor with increased levels of trust gained from audiences. The proposal presented in this paper holds potential for future studies focused on trustworthiness annotation, facilitating the measurement of trust-related features. Moreover, this research serves as a critical step towards understanding the foundations of trustworthiness in the development of synthetic agents that require perceived trustworthiness, particularly in domains involving negotiations or emergency situations where rapid data collection plays a pivotal role in saving lives.

Keywords

• Affective computing
• man–machine interaction (MMI)
• face expression
• gesture
• emotion
• trust modeling