Multi-Scale Generative Adversarial Network With Multi-Head External Attention for Image Inpainting

Abstract

Inpainting images that have sizable missing blocks presents a considerable challenge in terms of preserving visual consistency and attaining a convincing result. In this study, the multi-scale generative adversarial network with multi-head external attention for image inpainting (denoted as MGDAN) is proposed. First, an adaptive weight style loss is designed into the generator of multi-scale generative adversarial networks (GAN) to guide the inpainting of the style and structure in the image inpainting, which improves the inpainting effect of image contour and emoticon. Second, the Adaptive Mix model is introduced to address the imbalance in generator and discriminator training by reducing the distance between the difficult and easy samples. This approach is intended to improve the overall performance of the network and drive high-quality image generation. Third, the generator and local discriminator in the image inpainting process are enhanced by the introduction of a multi-head external attention mechanism. This addition aims to capture the long-distance and multi-level dependent relationships between different areas of the image. It is proved beneficial for the generating clear geometric contours in the restored images and improving the overall global consistency of the inpainted results. By employing specific experimental methods, we find the optimal number of heads. The effectiveness of the proposed approach also is demonstrated through the extensive experiments conducted on public datasets.

Keywords

• Image inpainting
• adaptive mix
• external attention
• multi-scale GANs