Hyperspectral Image Super Resolution Based on Multiscale Feature Fusion and Aggregation Network With 3-D Convolution

Abstract

The spectral resolution of hyperspectral images (HSIs) is very high. Nevertheless, their spatial resolution is low due to various hardware limitations. Therefore, it is important to study HSI super resolution to improve their spatial resolution. In this article, for hyperspectral single-image super resolution, we propose a multiscale feature fusion and aggregation network with 3-D convolution (MFFA-3D) by cascading the MFFA-3D block. The MFFA-3D block includes a group multiscale feature fusion part and a multiscale feature aggregation part. In group multiscale feature fusion part, a novel group multiscale feature fusion method is proposed. Group feature fusion module and two-step multiscale module are proposed in multiscale feature aggregation part. In order to prevent spectral distortion, a spectral gradient loss function is proposed and combined with the mean square error loss function to form the final loss function. Since the proposed super-resolution (SR) network is a full 3-D convolutional network, our method can perform direct super-resolution transfer even if the number of the bands of test images is different from that of the training images. The experiments over simulated and real HSIs demonstrate the superiority of the proposed method in terms of qualitative and quantitative evaluation.

Keywords

• 3-D convolutional neural network
• group
• hyperspectral image (HSI)
• multiscale feature fusion
• super resolution (SR)