Listening system using nodding images of spoken words for the purpose of promoting utterances of the elderly

Abstract

For the elderly, the act of “speaking” is important to maintain their physical function. It is known that the listening with nods and giving responses are important to encourage utterances. Also, in elderly nursing homes, conversations are often started when they look at images. We have developed a speech-driven embodied entrainment character called “InterActor” which automatically performs communicative motions such as nodding based on voice input, and have demonstrated the effectiveness of nodding reaction. In this study, we propose a listening system that promotes utterances by recognizing the user’s speech contents by voice input, and presenting images related to the words in speech on the display, and giving nodding reaction from the listener’s InterActor and images. This system has a function to support the intention to listen by having the listener’s InterActor to turn around to the image at the time of display and to gaze it jointly. To verify the effectiveness of this system, a comparative experiment and a sensory evaluation experiment were conducted on care workers at an elderly nursing home under the two conditions, one with or without of nodding image, and another with or without of the word image from the utterance. As a result, it was confirmed that the listening effect by image presentation and nodding motion has an utterance promoting effect that makes it easier for the user to speak.

Keywords

• embodied communication
• entrainment
• utterance promotion
• nodding