Mouth-in-the-Door: The Effect of a Sound Image of an Avatar Intruding on Personal Space That Deviates in Position From the Visual Image

Abstract

In this paper, we examine the audiovisual experience in virtual reality (VR) service context that enables a more effective interaction between a user immersed in a virtual environment (VE) and an avatar as a store staff. By utilizing the characteristics of VE experiences, we find the effects of this unrealistic relationship between the visual and auditory positions of the avatar presented to the user variable rather than uniformly presented in the same position. In this study, we conducted an experiment to investigate how the positional deviation between the sound and visual images can be tolerated in VE, the effect of positional deviation on the interpersonal distance to the avatar, and the possibility of manipulating the impression of the avatar by deviating the sound image from the visual image. For the experiment, we prepared a space resembling a VE store and conducted proximity experiments with 16 gender-balanced participants and six types of avatars. By utilizing the superiority of visual information over auditory information revealed in the experiments, we constructed an interpersonal situation with an avatar playing the role of store staff in which only the sound image intruded into the user’s personal space, and we investigated users’ impressions of the avatar. We also investigated users’ impressions of the avatar. We found the following two phenomena in the experimental conditions where the positional difference was allowed: 1) Even when the positional difference was allowed, it caused an “uncanny valley”-like phenomenon that led to a decrease in rapport; and 2) In the conditions where the positional difference was allowed when the sound image was closer than the visual image to the participant, the rapport was greater with the avatar playing the role of the store staff. This phenomenon is similar to the “foot-in-the-door” phenomenon in which small unconscious consent (i.e., allowing a sound image to intrude on one’s personal space) leads to an improvement in the evaluation of the other person (i.e., the rapport with another person). The techniques proposed in this paper, such as the positional difference between the sound and visual images, significantly improve the value of the service experiences obtained through interaction with others in VE.

Keywords

• Interaction
• interpersonal service
• personal space
• service on metaverse
• ventriloquism effect