SCRN: Stepwise Change and Refine Network Based Semantic Distribution for Human Pose Transfer

Abstract

It is a challenging and meaningful task to achieve person image synthesis by guiding pose. However, two problems have existed in past work: inaccurate generated poses and inconsistency with the target texture. To address these issues, we propose the Stepwise Change and Refine Network (SCRN), a two-stage network that aims to transfer given person images to the target pose while generating more reasonable and closer-to-real results. In the first stage, coarse images are generated using a series of modules with the same structure called Coarse Blocks. This process gradually changes the pose to achieve better shape consistency with the target image. In the second stage, style features are extracted from the original image by distributing semantic information. These features are used to optimize the rough image to obtain the final generated image, resulting in better consistency with the appearance of the target image. Our proposed method preserves both the pose’s spatial features and the original image’s texture features. Furthermore, we introduce a new loss function to make the generated image more in line with human perception. Qualitative and quantitative experiments with state-of-the-art models demonstrate significant improvements in SSIM, FID, PSNR, and LPIPS, validating the superiority of our model.

Keywords

• Deep learning
• generative adversarial network
• human pose transfer
• skeleton based approach
• person image synthesis