A voice-driven embodied interaction system with response actions according to speech speed by the mora

Abstract

Focusing on the correlation between speech sounds and body movements, we developed a voice-driven embodied entrainment character called InterActor that automatically generates communicative motions from speech, and demonstrated the effectiveness of the system. We also developed a communication agent that responds appropriately to utterance contents by focusing on words in speech using speech recognition. However, these systems simply generate body motions from the burst-pause of speech. The response actions corresponding to speaker’s speech speeds were not considered. In this paper, we analyze the relationship between an individual’s speech activity and speech speed[mora/s] from various speech experiments with the aim of facilitating speech according to speaker’s speech characteristics. We develop a voice-driven embodied entrainment system which performs nodding with different speeds according to speech speeds based on speech recognition. Based on the results of the speech experiments, we conduct an online evaluation experiment using videos of multiple speakers with different speaking speeds using the robot as a platform to verify the effectiveness of the response actions generated by the system.

Keywords

• embodied interaction
• embodied communication
• entrainment
• speech speed
• speech activity
• nodding