Cascaded Homography-Constrained Local Feature Matching for Optical and SAR Images

Abstract

Due to significant nonlinear radiation distortion and inherent speckle noise in SAR images, reliable matching between optical and SAR images remains a critical challenge. Current deep learning-based methods for optical and SAR image matching typically enhance specific steps of traditional algorithms, such as feature detection or description, but lack a comprehensive end-to-end solution. In response, we propose an end-to-end cascaded homography-constrained image matching (CHCIM) method for optical and SAR images. First, we combine CNN and transformers to efficiently fuse and extract similar features between optical and SAR images. In addition, a bilateral matching and homography-based cascade matching strategy is introduced for supervision and inference, which first refines the matching range using homography constraints and then employs random uniform sampling to select candidate features for further refinement. Extensive experiments demonstrate that CHCIM significantly outperforms the state-of-the-art baselines (e.g., RIFT and LoFTR) in matching accuracy (91.90% versus 50.10% and 44.37%), achieving the highest scores in mean matching accuracy (66.18% versus 4.01% and 4.67%) and the number of correct matches (1378 versus 43 and 26). Furthermore, CHCIM is effective in weak texture scenarios and robust to large scale and rotation variations. The code will be publicly available at https://github.com/LJY-RS.

Keywords

• Deep learning
• feature matching
• multimodal image
• SAR