TC-HISRNet: Hyperspectral Image Super-Resolution Network Based on Contextual Band Joint Transformer and CNN

Abstract

Deep learning-based methods for hyperspectral image super-resolution (HISR) have recently achieved remarkable success. However, the contextual correlation and intrinsic properties of adjacent bands in hyperspectral images have not been fully exploited in previous works. Therefore, this article proposes a network framework (TC-HISRNet) that leverages the local feature extraction capability of convolutional neural networks (CNNs) in tandem with the global feature extraction ability of transformer, to thoroughly investigate the role of contextual information in HISR. TC-HISRNet is comprised of a two-stage feature encoding network, which combines a two-channel CNN and a transformer encoder. Within the CNN encoder, the joint utilization of the real-time attention fusion module and two/three-dimensional convolutions enables the preservation of spectral correlation between bands while facilitating the extraction of local features from hyperspectral images. On the other hand, the developed transformer encoder leverages joint adjacent band contextual information to capture long-range dependencies and interdependencies among the combined features. Extensive evaluation and comparison of three public datasets demonstrated that the proposed method outperformed the state-of-the-art super-resolution method, resulting in superior reconstruction of hyperspectral images.

Keywords

• Band contextual information
• convolutional neural network (CNN)
• hyperspectral image super-resolution (HISR)
• real-time attention
• transformer