Circle-Net: An Unsupervised Lightweight-Attention Cyclic Network for Hyperspectral and Multispectral Image Fusion

Abstract

Hyperspectral image (HSI) and multispectral image (MSI) fusion has the potential to significantly improve the quality and usefulness of data, leading to better decision-making and a more complete understanding of the observed scene. For HSI and MSI fusion, capturing matched pairs of HSI and MSI images is challenging. This hampers the pretraining of neural-network-based HSI–MSI fusion methods and yields unsatisfactory fusion results. A lightweight-attention (LA) cyclic network (Circle-Net) without pretraining using labeled data is constructed and applied to HSI–MSI fusion to alleviate this issue. Circle-Net consists of a coordinate feature fusion (CFF) network and a dual-attention decoder (DAD) network. Multiscale features collected from the DAD network are fused by the CFF network to derive a high-resolution HSI. Specifically, in the DAD network, skip connections in the encoder–decoder network are replaced by LAs, while polarized attention is used to guarantee efficient transfer of features between the encoder and decoder. In comparison with other methods, the experimental performance shows the superiority of the Circle-Net in both visual and quantitative performance.

Keywords

• Attention mechanism
• deep learning (DL)
• hyperspectral images (HSIs)
• image fusion
• multispectral images (MSIs)