Video-Based Martial Arts Combat Action Recognition and Position Detection Using Deep Learning

Abstract

Action recognition in martial arts can offer valuable insights for technicians, athletes, and coaches. Accurate action recognition can enhance performance analysis, inform training strategies, and improve decision-making processes by providing detailed evaluations of technique execution, movement patterns, and match dynamics. This can lead to more effective coaching, better athlete preparation, and a deeper understanding of competitive outcomes. Existing methods in human action recognition often struggle with challenges such as background clutter, occlusion, and variations in appearance and speed, particularly in dynamic combat scenarios. In this study, we proposed a novel Spatio-Temporal Hierarchical Keypoint Aggregation (ST-HKA) framework for martial arts combat action recognition and localization. The ST-HKA model effectively leverages a deep learning-based approach that treats human skeleton keypoints as 3D point clouds. Unlike conventional methods that use graph convolutional networks or appearance-based techniques, our approach adopts a point cloud paradigm to treat human keypoints as a 3D point cloud, significantly improving scalability and robustness against occlusion and variations in appearance. Additionally, we incorporate a weakly supervised spatio-temporal action localization mechanism using a Context-Aware Pooling Mechanism. The proposed model was evaluated on both the Kinetics Human-Action and Taekwondo datasets, demonstrating superior performance in recognizing complex martial arts actions. The ST-HKA model achieves a Top-1 Accuracy of 88.6% and an F1-score of 83.9% on the Kinetics dataset, and 88.7% accuracy and an F1-score of 84.4% on the Taekwondo dataset. The proposed model also exhibits higher precision in detecting precise temporal boundaries, as reflected by its strong performance in action localization tasks. These results highlight the effectiveness of ST-HKA in handling complex martial arts actions with high accuracy and robustness.

Keywords

• Deep learning
• vision transformer
• event detection
• video classification
• martial art