Efficient Spectral-Spatial Fusion With Multiscale and Adaptive Attention for Hyperspectral Image Classification

Abstract

In hyperspectral image (HSI) classification, convolutional neural networks (CNNs) are widely used due to their ability to leverage the rich spectral information across multiple bands. However, HSI classification still faces various challenges, including insufficient spectral-spatial representation, excessive redundant information, and difficulties in effectively integrating features of different scales, etc., which may lead to reduced classification accuracy. In order to reduce the computational cost and improve the classification accuracy of land cover categories, an efficient spectral-spatial fusion method (ESSF) is proposed, which is based on the following modules: a multiscale feature fusion module (MSFFM), an efficient adaptive spectral-spatial feature extraction module (EASSFEM), and a context-aware fusion network (CFN). First, the MSFFM utilizes CNNs to extract and fuse features from various scales to comprehensively capture detailed spectral information in HSIs. Second, the EASSFEM dynamically adjusts the feature extraction process to optimize the fusion and representation of spectral-spatial features. In addition, it incorporates an adaptive attention mechanism to enhance the focus on relevant spectral-spatial features. Finally, the CFN enhances the model's ability to understand contextual relationships within the images, thereby improving classification accuracy. Extensive experiments conducted on four public datasets (Houston2013, Botswana, WHU-Hi-HanChuan and WHU-Hi-HongHu) demonstrate that the proposed ESSF method significantly outperforms nine other state-of-the-art methods in terms of classification accuracy.

Keywords

• Attention mechanism
• convolutional neural network (CNN)
• hyperspectral image (HSI)
• multiscale feature