Pansharpening Using Unsupervised Generative Adversarial Networks With Recursive Mixed-Scale Feature Fusion

Abstract

Panchromatic sharpening (pansharpening) is an important technology for improving the spatial resolution of multispectral (MS) images. The majority of the models are implemented at the reduced resolution, leading to unfavorable results at the full resolution. Moreover, the complicated relationship between MS and panchromatic (PAN) images is often ignored in detail injection. For the mentioned problems, unsupervised generative adversarial networks with recursive mixed-scale feature fusion for pansharpening (RMFF-UPGAN) are modeled to boost the spatial resolution and preserve the spectral information. RMFF-UPGAN comprises a generator and two U-shaped discriminators. A dual-stream trapezoidal branch is designed in the generator to obtain multiscale information. Further, a recursive mixed-scale feature fusion subnetwork is designed. Perform a prior fusion on the extracted MS and PAN features of the same scale. A mixed-scale fusion is conducted on the prior fusion results of the fine scale and coarse scale. The fusion is executed sequentially in the aforementioned manner building a recursive mixed-scale fusion structure, and finally, generating key information. A compensation information mechanism is also designed for the reconstruction of key information to compensate for information. A nonlinear rectification block for the reconstructed information is developed to overcome the distortion induced by neglecting the complicated relationship between MS and PAN images. Two U-shaped discriminators are designed and a new composite loss function is defined. The presented model is validated using two satellite data and the outcomes reveal better than the prevalent approaches regarding both visual assessment and objective indicators.

Keywords

• Mixed-scale feature fusion
• pansharpening
• reconstructed information rectification
• U-shaped discriminator
• unsupervised generative adversarial networks