A speech-driven pupil response system with affective highlight by virtual lighting

Abstract

In face-to-face communication, talkers perceive various affects and emotions of other talkers as the eye-impressions such as highlight or dilated pupil. In the previous studies, focusing on the eye-impressions in social robots, various expression methods such as changing the LED color or blinking the LED were proposed and developed. However, it is difficult to generate the affective expressions with the highlight that has an optical reflection characteristic on the surface of the eyeball by using the LED. Therefore, in order to enhance the eye-impressions of social robots, it is expected to develop an expressible method with the affective highlight. In this study, we proposed an affective highlight to enhance the eye-impressions of social robots. This method introduces a virtual lighting in virtual space and can generate rich affective expressions by controlling various parameters of the virtual lighting such as brightness or colors. In addition, we developed a speech-driven pupil response system that applied the affective highlight to the pupil response interface which has been developed. This system can generate the affective expressions by the dilation of pupil response that is synchronized with the talker’s speech as well as the highlight expression with talker’s intentional input. The effectiveness of the developed system with the affective highlight was demonstrated by sensory evaluations.

Keywords

• non-verbal communication
• affective expression
• pupil response
• highlight
• virtual lighting