Super-Resolution Reconstruction of Remote Sensing Images Using Generative Adversarial Network With Shallow Information Enhancement

Abstract

The super-resolution (SR) reconstruction method based on deep learning can significantly improve the spatial SR of remote sensing images. However, the current methods make insufficient use of the remote context information and channel information in shallow feature extraction, resulting in the limited effect of SR reconstruction. This article proposed a new SR reconstruction model, SIEGAN, which uses generative adversarial network with shallow information enhancement to improve the effect of SR reconstruction of remote sensing images. Similar to other generative adversarial models, SIEGAN is composed of generator and discriminator. But SIEGAN enhances the generator's ability to extract shallow information by using three different scale convolution operations. Specifically, a depthwise convolution is used to extract the local context information of each band of the image. A depthwise dilation convolution is used to capture the remote context information in the image. Finally, a 1×1 convolution is used to extract the correlation features between different channels in remote sensing images. In addition, SIEGAN uses U-Net network as its discriminator to provide detailed feedback per pixel to the generator to improve the model's ability to identify image details. And the spectral–spatial total variation loss function is introduced to ensure the spectral–spatial reliability of the reconstructed images. The experimental results on Gaofen-1 data proved that compared with the state-of-the-art models, SIEGAN has achieved better SR reconstruction performance. Furthermore, the reconstructed images by SIEGAN demonstrate better performance in land cover classification.

Keywords

• Generative adversarial network (GAN)
• multiscale shallow information
• remote sensing images
• super-resolution (SR) reconstruction