BeatNet+: Real-Time Rhythm Analysis for Diverse Music Audio

Abstract

This paper presents a comprehensive study on real-time music rhythm analysis, covering joint beat and downbeat tracking for diverse kinds of music signals. We introduce BeatNet+, a two-stage approach to real-time rhythm analysis built on a previous state-of-the-art method named BeatNet. The main innovation of the proposed method is the auxiliary training strategy that helps the neural network model to learn a representation invariant to the amount of percussive components in the music. Together with other architectural improvements, this strategy significantly improves the model performance for generic music. Another innovation is on the adaptation strategies that help develop real-time rhythm analysis models for challenging music scenarios, including isolated singing voices and non-percussive music. Two adaptation strategies are proposed and experimented with using different neural architectures and training schemes. Comprehensive experiments and comparisons with multiple baselines are conducted, and results show that BeatNet+ achieves superior beat tracking and downbeat tracking F1 scores for generic music, isolated singing voices, and non-percussive audio, with competitive latency and computational complexity. Finally, we release beat and downbeat annotations for two datasets that are designed for other tasks, and revised annotations of three existing datasets. We also release the code repository and pre-trained models on GitHub.

Keywords

• real-time beat tracking
• downbeat tracking
• rhythm analysis
• singing voices
• non-percussive music
• beatnet
• beatnet+