Dimensional Expansion and Time-Series Data Augmentation Policy for Skeleton-Based Pose Estimation

Abstract

Human pose estimation has long been researched as a significant topic in computer vision. However, studies via deep learning models are insufficient because of the lack of 2D and 3D skeleton data in various domains. An augmentation technique is applied to solve the problem of data scarcity. Data augmentation techniques can improve the performance of an analytical model by increasing the amount of data. However, the model’s performance is degraded if the augmented results differ significantly from the actual distribution. Therefore, it is necessary to implement an optimized augmentation policy for image datasets. This study proposed a dimensional expansion and time-series data augmentation policy for pose estimation based on skeletons. The proposed method improves the model’s performance by using 3D skeleton data. The 3D skeleton data were preprocessed through an affine transformation. The data were augmented for dimensional expansion from 2D to 3D data. In addition, sampling was applied to the data in consideration of time-series features, and thus, the number of frames per unit time was redefined. Subsequently, part of the information was lost by cutout, and thus, the data size, rather than the data shape, was changed. Through the image and video augmentation policies and cutout expansion, search candidates for 3D time-series data augmentation policies were extracted from the number of cases generated through the combination of 16 skeleton augmentation policies, 11 probabilities, and ten intensities. Finally, 20 candidates were extracted, and the five best-performing policies were applied.

Keywords

• Data augmentation policy
• dimensional expansion
• pose estimation
• skeleton data