Multimodal Metronome — Rhythm game for musical instruments

Abstract

The goal of this project is to create an application that uses auditory, visual, and tactile elements to assist and guide musical instrument learners, using features available on the Nintendo Switch device and its paired “Joy-Con” detachable interface, to increase the novelty as well as intuitiveness of musical performance while stimulating user experience. By integrating visual music, spectral analysis, and motion capture, we obtain a metronome system that synchronizes visual, acoustic, and haptic display.