Hyperspectral Image Classification Based on 3-D Multihead Self-Attention Spectral–Spatial Feature Fusion Network

Abstract

Convolutional neural networks are a popular method in hyperspectral image classification. However, the accuracy of the models is closely related to the number and spatial size of training samples. Which relieve the performance decline by the number and spatial size of training samples, we designed a 3-D multihead self-attention spectral–spatial feature fusion network (3DMHSA-SSFFN) that contains step-by-step feature extracted blocks (SBSFE) and 3-D multihead-self-attention-module (3DMHSA). The proposed step-by-step feature extracted blocks relieved the declining-accuracy phenomenon for the limited number of training samples. Multiscale convolution kernels extract more spatial–spectral features in the step-by-step feature-extracted blocks. In hyperspectral image classification, the 3DMHSA module enhances the stability of classification by correlating disparate features. Experimental results show that 3DMHSA-SSFFN possesses a better classification performance than other advanced models through the limited number of balance and imbalance training data in three data.

Keywords

• Convolutional neural network (CNN)
• hyperspectral image classification
• multihead self-attention
• multiscale convolution