RPF: Reference-Based Progressive Face Super-Resolution Without Losing Details and Identity

Abstract

Face super-resolution involves generating a high-resolution facial image from a low-resolution one. It is, however, quite a difficult task when the resolution difference between input and output images is too large. In order to tackle this challenge, many approaches use generative adversarial networks that are pre-trained on a large facial image dataset, but they often generate fake details and distort the person’s original face, leading to a loss of identity. Hence, in this paper, we propose a progressive face super-resolution network, called RPF, to super-resolve a facial image without losing details and personal identity by progressively exploiting the same person’s high-resolution image as a reference image. First, we remove unnecessary detail information, such as hair and background, from the reference image, which may be different from the low-resolution input. Next, we align the high-resolution reference image to the low-resolution input image and blend them to generate a synthesized image. Finally, we refine the synthesized image to generate a faithful super-resolved image containing both details and identity information. Experimental results demonstrate that the proposed RPF algorithm outperforms recent state-of-the-art methods in terms of detail restoration and identity preservation, with improvements of 0.0098 and 0.0478 in LPIPS and ISC, respectively, on the CelebA-HQ dataset.

Keywords

• Face super-resolution
• reference-based super-resolution
• convolutional neural networks
• generative adversarial networks