Implicit Semantic Data Augmentation for Hand Pose Estimation

Abstract

Data augmentation is a well-known technique used for improving the generalization performance of modern neural networks. After the success of several traditional random data augmentation for images (including flipping, translation, or rotation), a recent surge of interest in implicit data augmentation techniques occurs to complement random data augmentation techniques. Implicit data augmentation augments training samples in feature space, rather than in pixel space, resulting in the generation of semantically meaningful data. Several techniques on implicit data augmentation have been introduced for classification tasks. However, few approaches have been introduced for regression tasks with continuous/structured labels, such as object pose estimation. Hence, we are motivated to propose a method for implicit semantic data augmentation for hand pose estimation. By considering semantic distances of hand poses, the proposed method implicitly generates extra training samples in feature space. We propose two additional techniques to improve the performance of this augmentation: metric learning and hand-dependent augmentation. Metric learning aims to learn feature representations to reflect the semantic distance of data. For hand pose estimation, the distribution of augmented hand poses can be regulated by managing the distribution of feature representations. Meanwhile, hand-dependent augmentation is specifically designed for hand pose estimation to prevent semantically meaningless hand poses from being generated (e.g., hands generated by simple interpolation between both hands). Further, we demonstrate the effectiveness of the proposed technique using two well-known hand pose datasets: STB and RHD.

Keywords

• Hand pose estimation
• data augmentation
• semantic learning
• feature learning