Speech-driven embodied entrainment character system with pupillary response

Abstract

We have developed a speech-driven embodied entrainment character called “InterActor” that has functions of both speaker and listener for supporting human interaction and communication. This character would generate communicative actions and movements such as nodding, body movements, and eyeball movements by using only speech input. In this study, we focus specifically on the pupillary responses related to human emotions in embodied interaction and communication. Pupillary responses in human face-to-face communication are analyzed using an embodied communication system with a line-of-sight measurement device. Based on the analysis results, we enhance the functions of the character and develop an advanced speech-driven embodied entrainment character system for conveying empathy. Using only speech input, this system changes the pupil size of characters based on the analysis. Through sensory evaluation, we perform experiments to determine the effects of the developed system. The results reveal that the system is effective in human interaction and communication.

Keywords

• human interaction
• nonverbal communication
• face-to-face communication
• line-of-sight
• pupillary response