Denoising Graph Autoencoder for Missing Human Joints Reconstruction

Abstract

Skeleton-based human action recognition (HAR) is being utilized in various fields like action classification and abnormal behavior detection. The accurate coordinates of the human joints are a crucial factor for the high performance in skeleton-based HAR. However, the missing joints caused by occlusion and invisibility result in performance degradation. Hence, in this paper, a missing joint reconstruction model is proposed to improve the performance of skeleton-based HAR. The proposed model, based on a denoising graph autoencoder (DGAE), regards missing joints as noise corrupted information and aims to reconstruct them to be close to their original coordinates. When the encoder of the proposed model compresses the noised input into a latent vector, a masking Laplacian matrix is introduced to reduce the effect of the missing joints’ features. The masking Laplacian matrix adjusts the effect of features between a missing joint and its adjacent joints by altering the weights of an adjacent matrix. In the decoder, a Laplacian matrix, which represents the connections among the joints, is utilized to reconstruct an output from the latent vector. The experiment result shows that the proposed model reconstructs the coordinates of missing joints with a marginal error. In addition, the performance of skeleton-based HAR is enhanced by reconstructing the missing joints.

Keywords

• Autoencoder
• graph convolution
• joint reconstruction
• pose estimation