Multimodal Technologies and Interaction (May 2025)

Methods and Findings in the Analysis of Alignment of Bodily Motion in Cooperative Dyadic Dialogue

  • Zohreh Khosrobeigi,
  • Maria Koutsombogera,
  • Carl Vogel

DOI
https://doi.org/10.3390/mti9060051
Journal volume & issue
Vol. 9, no. 6
p. 51

Abstract

Read online

This research analyses the temporal flow of motion energy (ME) in dyadic dialogues using alternating lagged correlation tests on consecutive windows and also Granger causality (GC) tests. This research considers both alternatives of lagged values, those of the more dominant party preceding those of the less and vice versa (with relative dominance independently determined) and labels the resulting lagged windows according to the category of correlation (CC) that holds (positive, negative or none, if the correlation is not significant). Similarly, windows are labeled in relation to the significance of GC (one party causing the other, mutual causation, or no causation). Additionally, occurrences of gestures or speech within windows by an interlocutor whose ME precedes are identified. Then, the ME GC labels are compared with labels derived from simple lagged correlation of ME values to identify whether GC or CC is more efficacious in highlighting which participant independent observers classify as the more dominant party, potentially the “leader” for the conversation. In addition, the association between speech, gestures, dominance, and leadership is explored. This work aims to understand how the distributions of these labels interact with independent perceptions of dominance, to what extent dominant interlocutors lead, and the extent to which these labels “explain” variation in ME within any dialogue. Here, the focus is on between speakers dynamics. It shows dominant speakers have measurable influence on their conversation partners through bodily ME, as they are more likely to lead motion dynamics, though moments of mutual influence also occur. While GC and lagged correlation both capture aspects of leadership, GC reveals directional influence, whereas correlation highlights behavioural alignment. Furthermore, ME contrast during speaking and interaction of ME and gestures indicate that bodily movement synchronisation is shaped not only by dominance but also by gesture types and speaking states: speech affects leadership more than gestures. The interactions highlight the multimodal nature of conversational leadership, where verbal and nonverbal modalities interact to shape dialogue dynamics.

Keywords