Spectral–Spatial Attention-Guided Multi-Resolution Network for Pansharpening

Abstract

Pansharpening is a technique that combines high-resolution panchromatic (PAN) images with low-resolution multispectral (MS) images to produce high-resolution MS (HRMS) images. Deep learning-based pansharpening have outperformed traditional methods on detail injection and spectral preserving. However, existing methods often directly learn the mapping between PAN, MS, and fused HRMS, without considering the spectral–spatial feature correlation in separate bands among PAN, low-resolution PAN (LRPAN), and MS. To address this limitation, we propose a novel network called spectral–spatial attention-guided multiresolution network (SSA-MRN). Initially, SSA-MRN incorporates LRPAN images to capture the intermediate features between MS and PAN images. It also uses the individual bands of MS to learn band-specific features. Based on the comprehensive features, the spectral–spatial attention integration (SSAI) module is introduced at various scales. SSAI leverages a dot-product attention mechanism to selectively enhance the associative spectral–spatial features between PAN images and MS images across different spectral bands. The features learned by the SSAI are progressively fused at each resolution to produce the final output. Experiments on two benchmark datasets are conducted at both reduced-resolution and full-resolution. Results demonstrate that our SSA-MRN significantly enhances pansharpening quality compared to five classical methods and four state-of-the-art deep learning-based methods.

Keywords

• Multi-resolution
• multi-spectral (MS) feature
• pansharpening
• spectral–spatial attention integration (SSAI)