Classification of Pilates Using MediaPipe and Machine Learning

Abstract

Pilates has established itself as a powerful instrument for a particular kind of rehabilitation therapy. Pilates can benefit people at all stages of life, especially in terms of lowering pain and disability, through enhancing physical performance and condition. It is now well established that Pilates is also likely to lead to secondary injuries if the exercise posture is not standard, so professional guidance and supervision is particularly important. This paper proposes classifying Pilates poses accurately using deep learning techniques. Ten Pilates poses are described, including Criss Cross, Rolling Ball, Spine Twist, High Plank to Pike, One Hundred, Climber, Squat, kneel Side Kick, Roll Up and Bird Dog. For each Pilates poses in this paper, 37 videos from various teachers and perspectives were gathered from the YouTube. The dataset consisted of 370 videos. The 3D human skeletal keypoints in the video frames are identified using MediaPipe in this study, and convolutional neural network(CNN) to extract features from the obtained human keypoints, combined with the Long Short-Term Memory(LSTM) algorithm to recognize Pilates poses in real time video. The model achieved an accuracy of 94.37% for the recognition of Pilates poses in single frame. An accuracy of 95.70% was achieved after polling predictions on 120 frames of video.

Keywords

• Pilates
• MediaPipe
• convolutional neural network
• long short-term memory