A Unified Framework From Face Image Restoration to Data Augmentation Using Generative Prior

Abstract

In the domain of data enhancement, image restoration and data augmentation are two tasks gaining increasing attention. Current image restoration models focus on improving clarity using pre-trained generative models, and data augmentation methods try to generate new samples with the help of generative models. These two related topics have long been studied completely separately. We propose a downstream-friendly restoration framework based on pre-trained generative models with the capability of data augmentation for face images. We carefully design our framework to achieve high fidelity when inheriting the generation ability from the pre-trained generator. To achieve this goal, we use a modified U-Net to predict the biases of latent codes and feature maps to guide the generator. We further propose to adopt linear interpolation as an approach to enriching the datasets for downstream tasks, especially for class-imbalanced tasks. Effectiveness of our method is demonstrated through experiments on three datasets and one downstream task.

Keywords

• Face image restoration
• data augmentation
• generative adversarial network
• linear interpolation
• latent inversion