Attention_FPNet: Two-Branch Remote Sensing Image Pansharpening Network Based on Attention Feature Fusion

Abstract

Inspired by the impressive achievements of convolutional neural networks in various computer vision tasks and the effective role of attention mechanisms, this article proposes a two-branch fusion network based on attention feature fusion (AFF) called Attention_FPNet to solve the pansharpening problem. We reconstruct the spatial information of the image in the high-pass filter domain and fully consider the spatial information in the multispectral (MS) and panchromatic (PAN) images. At the same time, the input PAN image and the upsampled MS image are directly transmitted to the reconstructed image through a long skip connection. The spectral information of the PAN and MS images is considered to improve the spectral resolution of the fused image. It also supplements the loss of spatial information that may be caused by network deepening. Moreover, an AFF method is used to replace the existing simple channel concatenation method commonly used in pansharpening, which fully considers the relationship between different feature maps and improves the fusion quality. Through experiments on image datasets acquired by the Pleiades, SPOT-6 and Gaofen-2 satellites, the results show that this method can effectively fuse PAN and MS images and generate a fused image and outperforms existing methods.

Keywords

• Attention feature fusion (AFF)
• convolutional neural network (CNN)
• image fusion
• pansharpening
• remote sensing