Two-Branch Generative Adversarial Network With Multiscale Connections for Hyperspectral Image Classification

Abstract

Hyperspectral image (HSI) classification has always drawn great attention in the field of remote sensing. Various deep learning models are in the ascendant and gradually applied to HSI classification. Nevertheless, limited-labeled and class-imbalanced datasets largely make the classifier prone to overfitting. To address the above problem, this article proposes a two-branch generative adversarial network with multiscale connections (TBGAN), which includes two generators to produce the spectral and spatial samples, respectively. Thereinto, the spectral generator is imbued with the self-attention mechanism to maximumly capture the long-term dependencies across the spectral bands. And meanwhile, an elaborated discriminator with two branches is devised in TBGAN for extracting the joint spectral-spatial features. Besides, the multiscale connections are placed between the discriminator and two generators to alleviate the instability problems caused by the inherently backward propagation of gradients in GAN. Furthermore, a feature-matching term is added to the loss function to prevent the generators from overtraining upon the current discriminator, thereby further improving the stability of the network. Experiments upon three benchmark datasets demonstrate that TBGAN achieves an extremely competitive classification accuracy and exerts lower sensitivity to the training sample size compared with several state-of-the-art methods.

Keywords

• Hyperspectral image classification
• generative adversarial network
• multiscale connections
• joint spectral-spatial features