A Novel Dual-Branch Pansharpening Network with High-Frequency Component Enhancement and Multi-Scale Skip Connection

Abstract

In recent years, the pansharpening methods based on deep learning show great advantages. However, these methods are still inadequate in considering the differences and correlations between multispectral (MS) and panchromatic (PAN) images. In response to the issue, we propose a novel dual-branch pansharpening network with high-frequency component enhancement and a multi-scale skip connection. First, to enhance the correlations, the high-frequency branch consists of the high-frequency component enhancement module (HFCEM), which effectively enhances the high-frequency components through the multi-scale block (MSB), thereby obtaining the corresponding high-frequency weights to accurately capture the high-frequency information in MS and PAN images. Second, to address the differences, the low-frequency branch consists of the multi-scale skip connection module (MSSCM), which comprehensively captures the multi-scale features from coarse to fine through multi-scale convolution, and it effectively fuses these multilevel features through the designed skip connection mechanism to fully extract the low-frequency information from MS and PAN images. Finally, the qualitative and quantitative experiments are performed on the GaoFen-2, QuickBird, and WorldView-3 datasets. The results show that the proposed method outperforms the state-of-the-art pansharpening methods.

Keywords

• deep learning
• pansharpening
• high-frequency component enhancement module (HFCEM)
• multi-scale skip connection module (MSSCM)
• dual-branch