A Hybrid Spectral Attention-Enabled Multiscale Spatial–Spectral Learning Network for Hyperspectral Image Superresolution

Abstract

Hyperspectral image superresolution (HSI-SR) has become an essential step of data preprocessing for tasks such as classification and change detection in remote sensing. For HSI-SR tasks, the state-of-the-art methods lie in how to learn effective spatial and spectral characteristics. More deeply mining the shallow and deep spatial–spectral features is vital for the performance improvement of HSI-SR. Hereby, we provide an end-to-end multiscale spatial–spectral network (M3SN) driven by a hybrid spectral attention mechanism (HSAM) for HSI-SR, aiming to fully dig shallow and deep spatial–spectral characteristics. Precisely, considering the importance of shallow spatial–spectral features at early stages, a multiscale information block consisting of a 3-D convolution, three parallel 2-D convolutions with different scale sizes, and SE-Net is first designed to extract informative multiscale shallow spatial–spectral features and recalibrate the channel weights of features. Then, a dual-path multiscale spatial–spectral feature block is set up to explore the deep spatial and spatial–spectral features. While one path using 2-D convolutions extracts spatial features, the other path employing 3-D-Res2Net as well as an updated HSAM module mines multiscale deep spatial–spectral features. Finally, we design a multiscale fusion block based on the channel reduction-scaling operation to fuse the extracted hierarchical feature maps for the final reconstruction. It is demonstrated that the M3SN outperforms existing methods by extensive experiments on four publicly available hyperspectral datasets.

Keywords

• Convolutional neural network (CNN)
• dual-path framework (DPF)
• hybrid spectral attention mechanism (HSAM)
• hyperspectral image superresolution (HSI-SR)
• multiscale spatial–spectral features