Main–Sub Transformer With Spectral–Spatial Separable Convolution for Hyperspectral Image Classification

Abstract

Due to their spatial and spectral information, hyperspectral images are frequently used in various scientific and industrial fields. Recent developments in hyperspectral image classification have revolved around the use of convolutional neural networks and transformers, which are capable of modeling local and global data. However, most of the backbone networks of existing methods are based on 3-D convolution, which has high complexity in network structure. Moreover, local information and global information are extracted through different modules, and the coupling relationship between the two types of information is weak. To address the above-mentioned issues, we propose a method named main–sub transformer network with spectral–spatial separable convolution method (MST-SSSNet), which includes two key modules: the spectral–spatial separable convolution (SSSC) module and the main–sub transformer encoder (MST) module. The SSSC module uses the proposed spectral–spatial separable convolution, reducing network parameters and efficiently extracting local features. The MST module adds the designed subtransformer in front of the conventional transformer encoder (main transformer). It assists the main-transformer encoder to establish global correlation by learning local information. The WHU-Hi dataset can be used as a benchmark dataset for precise crop classification and hyperspectral image classification research. MST-SSSNet is shown to deliver better classification performance than current state-of-the-art methods on the datasets.

Keywords

• Convolution neural networks (CNNs)
• hyperspectral image (HSI) classification
• main–sub transformer encoder (MST)
• spectral–spatial separable convolution (SSSC)