Sensing the Inside Out: An Embodied Perspective on Digital Animation Through Motion Capture and Wearables

Abstract

Over the last few decades, digital technology has played an important role in innovating the pipeline, techniques, and approaches for creating animation. Sensors for motion capture not only enabled the incorporation of physical human movement in all its precision and expressivity but also created a field of collaboration between the digital and performing arts. Moreover, it has challenged the boundaries of cinematography, animation, and live action. In addition, wearable technology can capture biosignals such as heart rate and galvanic skin response that act as indicators of the emotional state of the performer. Such metrics can be used as metaphors to visualise (or sonify) the internal reactions and bodily sensations of the designed animated character. In this work, we propose a framework for incorporating the role of the performer in digital character animation as a real-time designer of the character’s affect, expression, and personality. Within this embodied perspective, sensors that capture the performer’s movement and biosignals are viewed as the means to build the nonverbal personality traits, cues, and signals of the animated character and their narrative. To do so, following a review of the state of the art and relevant literature, we provide a detailed description of what constitute nonverbal personality traits and expression in animation, social psychology, and the performing arts, and we propose a workflow of methodological and technological toolstowardsan embodied perspective for digital animation.

Keywords

• digital animation
• biosignals
• sensors
• motion capture
• nonverbal communication
• character design