IFF-Net: Irregular Feature Fusion Network for Multimodal Remote Sensing Image Classification

Abstract

In recent years, classification and identification of Earth’s surface materials has been a challenging research topic in the field of earth science and remote sensing (RS). Although deep learning techniques have achieved some results in remote sensing image classification, there are still some challenges for multimodal remote sensing data classification, such as information redundancy between multimodal remote sensing images. In this paper, we propose a multimodal remote sensing data classification method IFF-Net based on irregular feature fusion, called IFF-Net. The IFF-Net architecture utilizes weight-shared residual blocks for feature extraction while maintaining the independent batch normalization (BN) layer. During the training phase, the redundancy of the current channel is determined by evaluating the judgement factor of the BN layer. If this judgment factor falls below a predefined threshold, it indicates that the current channel information is redundant and should be substituted with another channel. Sparse constraints are imposed on some of the judgment factors in order to remove extra channels and enhance generalization. Furthermore, a module for feature normalization and calibration has been devised to leverage the spatial interdependence of multimodal features in order to achieve improved discrimination. Two standard datasets are used in the experiments to validate the effectiveness of the proposed method. The experimental results show that the IFF-NET method proposed in this paper exhibits significantly superior performance compared to the state-of-the-art methods.

Keywords

• classification
• IFF-Net
• multimodal
• remote sensing (RS)
• BN
• judgment factor