Deep learning‐based action recognition with 3D skeleton: A survey

Abstract

Abstract Action recognition based on 3D skeleton data has attracted much attention due to its wide application, and it is one of the most popular research topics in computer vision. The 3D skeleton data is an effective representation of motion dynamics and is not easily affected by light, scene variation, etc. Previous research on action recognition has mainly focused on video or RGB data methods. In recent years, the advantages of combining skeleton data and deep learning have been gradually demonstrated, many impressive methods have been proposed, especially GCN‐based methods. In this survey, we first introduce the development process of 3D skeleton‐data action recognition and the classification of graph convolutional network, then introduce the commonly used NTU RGB + D and NTU RGB + D 120 datasets. Finally, a detailed review of existing variants of three mainstream technologies is provided based on deep learning and their performance was compared from three dimensions. To the best of our knowledge, this is the first research to integrate the research of GCN‐based method and its various evolutionary methods. Comparative investigation of existing variants of research in action‐recognition task from different perspectives is made, a generic framework is described, state‐of‐the‐art practices are summarized, and the emerging trends of this topic are explored.

Keywords

• computer vision
• feature extraction
• gesture recognition
• image classification
• image motion analysis
• image recognition