Music-Driven Synchronous Dance Generation Considering K-Pop Musical and Choreographical Characteristics

Abstract

Generating dance movements from music has been considered a highly challenging task, as it requires the model to comprehend concepts from two different modalities: audio and video. However, recently, research on dance generation based on deep learning has been actively conducted. Existing dance generation researches tend to focus on generating dances in limited genres or for single dancer, so when K-pop music that mixes multiple genres was applied to existing methods, they failed to generate dances of various genres or group dances. In this paper, we propose the K-pop dance generation model in an autoregressive manner, a system designed to generate two-person synchronous dances based on K-pop music. To achieve this, we created a dataset by collecting videos of multiple dancers simultaneously dancing to K-pop music and dancing in various genres. Generating synchronous dances has two meanings: one is to generate a dance that goes well with the input music and dance when both are given, and the other is to simultaneously generate multiple dances that match the given music. We call them secondary dance generation and group dance generation, respectively, and designed the proposed model, which can perform both two generation methods. In addition, we would like to propose additional learning methods to make a model that better generates synchronous dances. To assess the performance of the proposed model, both qualitative and quantitative evaluations are conducted, proving the effectiveness and suitability of the proposed model when generating synchronous dances for K-pop music.

Keywords

• Synchronous dance generation
• K-pop group dance generation
• autoregressive model
• multi-step learning