Progressive Reconstruction Network With Adaptive Frequency Adjustment for Pansharpening

Abstract

Pansharpening aims to reconstruct the high-resolution multispectral (HR-MS) image by fusing the low-resolution multispectral (LR-MS) image and the panchromatic (PAN) images. The effectiveness of pansharpening algorithms highly relies on the fusion mechanism of two images with different modality. In this article, we propose a novel progressive reconstruction network (PRNet) with adaptive frequency adjustment to address the spatial and frequency discrepancies between LR-MS and PAN images. Aiming to simultaneously capture contextual information and frequency characteristics, our PRNet fuses the LR-MS and PAN features in biscale frequency domains. Specifically, we use the wavelet transform to decompose the PAN features into biscale coefficients, dubbed as coarse-scale and fine-scale coefficients, respectively. Initially, the LR-MS features are fused into the coarse-scale coefficients first, and then the coarse-scale fused results are used to refine the fine-scale coefficients, leading to the progressive reconstruction mechanism. Additionally, to mitigate aliasing between LR-MS features and PAN coefficients across multiple frequency domains, we propose the adaptively frequency adjustment module to control the fusion weights and recalibrate the frequency responses of the reconstruct features. Experimental evaluations on reduced-resolution and full-resolution pansharpening datasets demonstrate that our method outperforms state-of-the-art pansharpening methods quantitatively and qualitatively, affirming its effectiveness. The code will be released soon.

Keywords

• Biscale features
• frequency module
• image fusion
• pansharpening
• wavelet transform