Global–Local Multigranularity Transformer for Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification is a challenging task in remote sensing applications, aiming to determine the category of each pixel by utilizing rich spectral and spatial information in HSI. Convolutional neural networks (CNNs) have been effective in processing HSI data by extracting local features, but they are deficient in capturing global contextual information. Recently, transformer has become proficient in attending to global information due to their self-attention mechanisms, yet they may fall short in capturing multiscale features of HSI. To address these limitations, a global–local multigranularity transformer (GLMGT) network is proposed for HSI classification. The GLMGT combines CNN with the transformer to comprehensively capture multigranularity spectral and spatial features across global and local scales. Specifically, we introduce a multigranularity spatial feature extraction block to extensively extract spatial information at different granularities, including multiscale local spatial features and global spatial features. In addition, we introduce a multigranularity spectral feature extraction block to fully leverage spectral information across different granularities. The validity of the proposed method is demonstrated through experimental validation using seven publicly available datasets, which include two Chinese satellite hyperspectral datasets (ZY1-02D Huanghekou and GF-5 Yancheng) and one UAV-based hyperspectral dataset.

Keywords

• CNN
• multigranularity
• transformer
• hyperspectral image (HSI) classification