Integrating Spectral and Spatial Bilateral Pyramid Networks for Pansharpening

Abstract

Multispectral (MS) image pansharpening is a significant technology for remote sensing image analysis, aiming to restore a high-resolution MS image by merging a high-resolution panchromatic image with a low-resolution MS image. While the convolutional neural networks (CNNs) have garnered considerable attention for their exceptional fusion capabilities in recent years, the existing CNN-based methods cannot effectively integrate spectral–spatial information. In this article, we introduce a novel MS pansharpening framework that integrates spectral and spatial networks in a bilateral pyramid form, allowing for the extraction of hierarchical spectral–spatial information. The proposed reduced residual dense (RRD) module serves as the fundamental building block of the spatial network. The RRD module gradually reduces the dimension of feature maps and employs the concatenation of RRD with global residual learning for comprehensive feature representation. In the spectral network, we present a cooperative attention fusion module to further enhance the correlation between spectral and spatial features. Through extensive experiments conducted on benchmarking simulated and real datasets, our proposed framework consistently outperforms state-of-the-art methods, demonstrating its effectiveness in MS image pansharpening applications.

Keywords

• Attention mechanism
• bilateral network
• convolution neural network (CNN)
• image fusion
• pansharpening