Effect of a Hearer's Politeness on Multimodal Behaviors in Speech

Abstract

This paper investigates the effect of a hearer's attitude toward a speaker's multimodal behaviors. Twenty-one university students, nine males and twelve females, participated in this experiment as speakers. Two females, aged 22 years old, performed in the role of hearers. The experiment was carried out under two conditions regarding the hearer's attitude: polite and impolite conditions. The polite hearer nodded, produced back-channel response, and gazed at the speaker, while the impolite hearer did these actions to a lesser extent. Under each condition, the speakers talked about their own experiences to the two different hearers through a within-participants design. The speech, gaze, and body motion of speakers were recorded with a video camera. In addition, the communication skills of speakers, including expressivity, sensitivity, regulation, assertiveness, and responsiveness skills, were measured after the experiment. The speakers' silent pauses, filled pauses, gaze aversion, and representational gestures were compared between the hearers' two politeness conditions. The relationship between the multimodal behaviors and the speakers' communication skills was also analyzed. The results suggest that the speakers' silent pauses, filled pauses, and gaze aversion decreased in the polite hearer condition, while representational gestures increased. From the communication-skills test, the speakers with higher expressivity, sensitivity, and regulation skills but lower assertiveness and responsiveness skills show longer silent pauses under the impolite hearer condition. Moreover, we discuss how the implications of our findings can enhance the relationship between social robots and people.

Keywords

• politeness
• multimodal behaviors
• communication skills
• narrative interaction