Multimodal Drumming Education Tool in Mixed Reality

Abstract

First-person VR- and MR-based Action Observation research has thus far yielded both positive and negative findings in studies observing such tools’ potential to teach motor skills. Teaching drumming, particularly polyrhythms, is a challenging motor skill to learn and has remained largely unexplored in the field of Action Observation. In this contribution, a multimodal tool designed to teach rudimental and polyrhythmic drumming was developed and tested in a 20-subject study. The tool presented subjects with a first-person MR perspective via a head-mounted display to provide users with visual exposure to both virtual content and their physical surroundings simultaneously. When compared against a control group practicing via video demonstrations, results showed increased rhythmic accuracy across four exercises. Specifically, a difference of 239 ms (z-ratio = 3.520, p < 0.001) was found between the timing errors of subjects who practiced with our multimodal mixed reality development compared to subjects who practiced with video, demonstrating the potential of such affordances. This research contributes to ongoing work in the fields of Action Observation and Mixed Reality, providing evidence that Action Observation techniques can be an effective practice method for drumming.

Keywords

• mr)%22%2C%22default_operator%22%3A%22AND%22%7D%7D%7D"> mixed reality (<span style="font-variant: small-caps">mr</span>)
• multimodal interfaces
• motor skill learning
• ao)%22%2C%22default_operator%22%3A%22AND%22%7D%7D%7D"> Action Observation (<span style="font-variant: small-caps">ao</span>)
• vc)%22%2C%22default_operator%22%3A%22AND%22%7D%7D%7D"> Virtual Co-embodiment (<span style="font-variant: small-caps">vc</span>)
• merged reality