A dual-branch multi-feature deep fusion network framework for hyperspectral image classification

Abstract

AbstractRecent research has proven that deep learning models are effective at mining the rich spectral-spatial information of hyperspectral images and attaining high classification performance. In this study, an end-to-end two-branch multi-feature deep fusion network framework (MFDFN) is proposed, combining object-oriented features of hyperspectral images after multi-scale super-pixel segmentation and original image point-image element neighbourhood block features, and designing different network modules in the upper and lower branches for different features in order to extract more distinguishable and advantageous features for classification. For the second part, a series spectral-spatial feature learning modules and augmentation modules for spatial feature learning are being developed to learn these features in a continuous manner while also effectively fusing features from various depth layers. It also uses a variety of techniques (BN, Dropout, residual join, etc.) to speed up the model’s convergence, avoid overfitting, and improve the model’s generalization performance. In addition, experiment results on the Indian Pines, University of Pavia, Tea, and Salinas datasets reveal that the proposed MFDFN framework outperforms current deep-learning-based methods in terms of comprehensive classification effectiveness evaluation.

Keywords

• Hyperspectral image classification
• double branch network structure
• superpixel segmentation
• convolutional neural network (CNN)
• attention mechanism
• residual network