HE-CycleGAN: A Symmetric Network Based on High-Frequency Features and Edge Constraints Used to Convert Facial Sketches to Images

Abstract

The task of converting facial sketch images to facial images aims to generate reasonable and clear facial images from a given facial sketch image. However, the facial images generated by existing methods are often blurry and suffer from edge overflow issues. In this study, we proposed HE-CycleGAN, a novel facial-image generation network with a symmetric architecture. The proposed HE-CycleGAN has two identical generators, two identical patch discriminators, and two identical edge discriminators. Therefore, HE-CycleGAN forms a symmetrical architecture. We added a newly designed high-frequency feature extractor (HFFE) to the generator of HE-CycleGAN. The HFFE can extract high-frequency detail features from the feature maps’ output, using the three convolutional modules at the front end of the generator, and feed them to the end of the generator to enrich the details of the generated face. To address the issue of facial edge overflow, we have designed a multi-scale wavelet edge discriminator (MSWED) to determine the rationality of facial edges and better constrain them. We trained and tested the proposed HE-CycleGAN on CUHK, XM2VTS, and AR datasets. The experimental results indicate that HE-CycleGAN can generate higher quality facial images than several state-of-the-art methods.

Keywords

• facial sketches to images
• cycle-consistency adversarial network
• high-frequency feature extractor
• multi-scale wavelet edge discriminator