Multimodal Technologies and Interaction (Sep 2020)

ODO: Design of Multimodal Chatbot for an Experiential Media System

  • Ravi Bhushan,
  • Karthik Kulkarni,
  • Vishal Kumar Pandey,
  • Connor Rawls,
  • Brandon Mechtley,
  • Suren Jayasuriya,
  • Christian Ziegler

DOI
https://doi.org/10.3390/mti4040068
Journal volume & issue
Vol. 4, no. 4
p. 68

Abstract

Read online

This paper presents the design of a multimodal chatbot for use in an interactive theater performance. This chatbot has an architecture consisting of vision and natural language processing capabilities, as well as embodiment in a non-anthropomorphic movable LED array set in a stage. Designed for interaction with up to five users at a time, the system can perform tasks including face detection and emotion classification, tracking of crowd movement through mobile phones, and real-time conversation to guide users through a nonlinear story and interactive games. The final prototype, named ODO, is a tangible embodiment of a distributed multimedia system that solves several technical challenges to provide users with a unique experience through novel interaction.

Keywords