Human action recognition based on tensor shape descriptor

Abstract

Human action recognition is an important task. This study presents an efficient framework for recognising action with a 3D skeleton kinematic joint model in less computational time for practical usage. First, a tensor shape descriptor (TSD) is proposed in this study, which takes advantage of the spatial independence of body joints, avoids a lot of difficult problem of the explicit motion estimation required in traditional methods, reserves the spatial information of each frame. Thus, the new TSD is a complete and view‐invariant descriptor. Second, a novel tensor dynamic time warping (TDTW) method is proposed to measure joint‐to‐joint similarity of 3D skeletal body joints locally in the temporal extent, which is implemented by extending DTW to that of two multiway data arrays (or tensors). Then, a multi‐linear projection process is employed to map the TSD to a low‐dimensional tensor subspace, which is classified by the nearest neighbour classifier. The experiment results on the public action data set (MSR‐Action3D) and motion capture data set (CMU_Mocap) show that the proposed method can achieve a comparable or better performance in recognition accuracy compared with the state‐of‐the‐art approaches.

Keywords

• human action recognition
• tensor shape descriptor
• 3D skeleton kinematic joint model
• TSD
• explicit motion estimation
• novel tensor dynamic time warping method