Multiscale Spectral–Spatial Attention Residual Fusion Network for Multisource Remote Sensing Data Classification

Abstract

The joint of multisource remote sensing (RS) data for land cover classification has become a popular research topic. Although studies have shown that the fusion of multisource data can improve the accuracy of classification, the current limitation lies in the inadequate exploitation of information, resulting in spectral confusion categories overlapping, and varying visual differences within the same category. To address these problems, this article proposes a multiscale spectral–spatial attention residual fusion network (MSSARFNet) that aims to enhance the classification performance of multisource RS data through effective spectral–spatial feature extraction and fusion. Specifically, three modules are designed: the multiscale spectral attention residual module (MSpeARM), the multiscale spatial attention residual module (MSpaARM), and multiscale convolution fusion (MCF) module. First, we divide the channels of the features into multiple paths and apply spectral–spatial attention mechanisms on each path. To further enhance connectivity, convolutional operation is utilized to establish connections between different paths, thus forming MSpeARM and MSpaARM. The MSpeARM suppresses redundant features and enhances effective features along the channel dimension to better differentiate spectral-confused categories. The MSpaARM highlights that different visual patterns of objects in the same category can mutually reinforce each other by weighting all positional features, regardless of their spatial differences. Second, to fuse these two sets of features, the MCF module is designed to learn multilevel semantic features and enhance fusion at a granular level. Experimental evaluations on three RS datasets demonstrate that the proposed method achieves excellent classification performance, indicating the effectiveness of MSSARFNet.

Keywords

• Multiscale spectral–spatial attention residual fusion network (MSSARFNet)
• multisource data