Multi-Reception and Multi-Gradient Discriminator for Image Inpainting

Abstract

Many deep learning methods for image inpainting rely on the encoder-decoder architecture to estimate missing contents. When guidance information from uncorrupted regions cannot be adequately represented or utilized, the encoder may have difficulty handling the rich surrounding or background pixels, and the decoder could not recover visually sophisticated or realistic content. This paper proposes an effective multi-scale optimization network to alleviate these issues and generate coherent results with fine details. It adaptively encodes multi-receptive fields feature maps and puts multi-scale outputs into a discriminator to guide training. Specifically, we propose a Multi-Receptive feature maps & masks Selective Fusion (MRSF) operator that can adaptively extract features in different receptive fields to handle sophisticated destroyed images. Then a multi-gradient discriminator (MGD) module uses the intermediate features of the discriminator to guide the generator to produce results with natural textures and semantically real contents. Experiments on several benchmark datasets demonstrate that the proposed method can synthesize more realistic and coherent image content.

Keywords

• Image inpainting
• multi-receptive feature maps and masks selective fusion
• multi-gradient discriminator