Image super‐resolution based on self‐similarity generative adversarial networks

Abstract

Abstract Self‐attention has been successfully leveraged for long‐range feature‐wise similarities in deep learning super‐resolution (SR) methods. However, most of the SR methods only explore the features on the original scale, but do not take full advantage of self‐similarities features on different scales especially in generative adversarial networks (GAN). In this paper, self‐similarity generative adversarial networks (SSGAN) are proposed as the SR framework. The framework establishes the multi‐scale feature correlation by adding two modules to the generative network: downscale attention block (DAB) and upscale attention block (UAB). Specifically, DAB is designed to restore the repetitive details from the corresponding downsampled image, which achieves multi‐scale feature restoration through self‐similarity. And UAB improves the baseline up‐sampling operations and captures low‐resolution to high‐resolution feature mapping, which enhances the cross‐scale repetitive features to reconstruct the high‐resolution image. Experimental results demonstrate that the proposed SSGAN achieve better visual performance especially in the similar pattern details.

Keywords

• Optical, image and video signal processing
• Computer vision and image processing techniques
• Neural nets