Improving Human Pose Estimation With Self-Attention Generative Adversarial Networks

Abstract

Human pose estimation in images is challenging and important for many computer vision applications. Large improvements in human pose estimation have been achieved with the development of convolutional neural networks. Even though, when encountered some difficult cases even the state-of-the-art models may fail to predict all the body joints correctly. Some recent works try to refine the pose estimator. GAN (Generative Adversarial Networks) has been proved to be efficient to improve human pose estimation. However, GAN can only learn local body joints structural constrains. In this paper, we propose to apply Self-Attention GAN to further improve the performance of human pose estimation. With attention mechanism in the framework of GAN, we can learn long-range body joints dependencies, therefore enforce the entire body joints structural constrains to make all the body joints to be consistent. Our method outperforms other state-of-the-art methods on two standard benchmark datasets MPII and LSP for human pose estimation. Our code is available at: https://github.com/idotc/Hg-SAGAN.

Keywords

• Human pose estimation
• convolutional neural networks
• stacked hourglass networks
• self-attention GAN