Guidance Information Assisted Reconstruction of Masked Faces

Abstract

The application of deep learning to face inpainting has led to the development of a variety of methods. Guidance information, including edges, landmarks, segmentation maps, and sketches, has been increasingly incorporated alongside input images to achieve more stable structural restoration in face inpainting. Most of the methods utilizing guidance information predict the occluded part’s guidance information and subsequently incorporate it as part of the inputs to the inpainting module. However, these methods can adversely affect the final reconstruction result if the guidance information is incorrectly predicted. Therefore, we propose a face reconstruction method using guidance information outside of the occluded area. Additionally, we employed a conventional edge detection method and downsized the overall model structure due to the substantial computational expenses associated with utilizing deep neural networks for generating guidance information. During experiments conducted on the CelebA-HQ datasets, the proposed method demonstrated superior performance compared to other approaches, as evidenced by higher values in the SSIM, PSNR, and FID metrics.

Keywords

• Image inpainting
• face inpainting
• guidance information
• deep learning