A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability

Abstract

Synthetic Aperture Radar (SAR) and optical image fusion can gain images with abundant spatial, scattering, and spectral information. However, the current fusion methods are still faced with the challenge of spatial distortion, detail blur and inadequate use of polarization information. Thus, this paper proposes a Sentinel-1/2 images fusion framework based on multiscale transform (MST) and generative adversarial network (GAN) and designs a polarization-spectrum fusion scheme for multi-polarization SAR and multispectral optical images. Firstly, the framework utilizes GAN to fuse low-frequency coefficients of MST to effectively improve the contrast of fusion results. Secondly, based on this framework, the nonsubsampled shearlet transform (NSST) algorithm of MST is selected to decompose input images. The parameter-adaptive pulse coupled neural network (PAPCNN) model is then utilized for fusing high-frequency coefficients from NSST to reduce spatial distortion while effectively preserving the texture details of images. Thirdly, the “polarization-spectrum” fusion scheme of “VV polarization + B band, VH polarization + NIR band and VV/VH + G band” is designed to effectively improve the distinguishability of water bodies and different vegetation. Finally, the subjective and objective evaluation results of our method are significantly superior to other seven methods. In addition, our method is extended to 14 terrestrial biomes. The experimental results show that fused images fully integrate scattering, texture details and spectral information of Sentinel-1/2 images, compared with ESAWorldCover2020 products, which helps to better distinguish various ground objects in different scenarios.

Keywords

• SAR
• Image fusion
• Generative adversarial network
• NSST
• PAPCNN