MPEFNet: Multilevel Progressive Enhancement Fusion Network for Pansharpening

Abstract

Remote sensing image fusion is a key technique to fuse low spatial resolution multispectral (MS) images with high spatial resolution panchromatic (PAN) images to obtain high spatial resolution multispectral images. However, many existing fusion algorithms typically perform a single upsampling on the MS image to match its spatial resolution with that of the PAN image, and subsequently output the fused image through steps of feature extraction, fusion, and decoding. This single-stage fusion approach not only fails to fully utilize the low-frequency and high-frequency spatial information in the PAN image, but also leads to inadequate extraction of internal spatial and spectral information in the original MS image, resulting in problems such as blurring, artifacts, and incomplete spectral information recovery in the fused image. To address these issues, this article proposed a multilevel progressive enhancement fusion network. To fully fuse the spatial and spectral information of different resolution images, this article employs a three-stage network structure. The high preserving block is used to alleviate spatial detail distortion and spectral information loss caused by upsampling. Bands aggregation module and spatial aggregation module are used to refine the feature extraction module's spectral and spatial detail features. Meanwhile, the enhanced fusion module further performs self-enhancement fusion on the refined features, as well as mutual-enhancement fusion with the original information. The method is superior to the comparison method by qualitative analysis and quantitative comparison on the IKONOS and WorldView-2 datasets.

Keywords

• Convolutional neural network (CNN)
• image fusion
• multilevel
• multispectral (MS) image
• pansharpening