PI-ADFM: Enhancing Multimodal Remote Sensing Image Matching Through Phase-Integrated Aggregated Deep Features

Abstract

Geometric distortions and significant nonlinear radiometric differences in multimodal remote sensing images (MRSIs) introduce substantial noise in feature extraction. Single-branch convolutional neural networks fail to capture global image features and integrate local and global information effectively, yielding deep descriptors with low discriminability and limited robustness. Moreover, the lack of comprehensive training data further limits the network's performance, which poses a formidable challenge to existing matching methods in securing adequate and evenly distributed corresponding points. This article proposes a novel method called phase-integrated aggregated deep feature matching (PI-ADFM), designed to address these challenges. Initially, a phase structure feature detector is introduced, which amalgamates the structural attributes and phase information of images to distill keypoints that are highly repeatable and exhibit minimal redundancy. Subsequently, an attention-based multilevel feature interaction and aggregation module is crafted to encapsulate a comprehensive representation of both local and global features of keypoints. This is followed by the integration of a dense feature fusion module designed to sift through and amalgamate key features, thereby capturing highly discriminative deep semantic features that serve as descriptors for similarity measures. Finally, a multilevel outlier removal strategy is proposed to effectively reduce mismatches. Experimental results substantiate that, in juxtaposition with state-of-the-art methods, the PI-ADFM method has significantly augmented the count of matches for optical-infrared and optical synthetic aperture radar images by a factor of at least 1.7 and 3.7, respectively, while concurrently enhancing the accuracy by a minimum of 10% and 6%, respectively. These enhancements markedly bolster the robustness and reliability of MRSI matching endeavors.

Keywords

• Deep learning
• feature interaction and aggregation
• feature matching
• multimodal images
• phase structure