SIFNet: A self-attention interaction fusion network for multisource satellite imagery template matching

Abstract

Multisource satellite images provide abundant and complementary earth observations, while nonlinear radiometric and geometric distortions (such as scale and rotation variations) between these multimodal images pose remarkable challenges for further remote sensing applications, such as change detection. We therefore proposed a template matching algorithm based on self-attention interactive fusion network, named SIFNet, to align multisource satellite images. First, a feature pyramid network was first conducted to extract multiscale features for each pixel, with the template and reference images as the inputs. Then, the extracted features were fused by self-attention layers in Transformer for information interaction. Third, the similarity and semantic matching loss functions were developed to convert satellite imagery registration into regression task, allowing SIFNet aligning multimodal patch images more efficiently based on point-to-point correspondence, instead of globally searching extremums as previous matching strategies did. We performed experiments based on four multimodal datasets (i.e. Google, GF-2, Landsat-8 and optical-SAR images) with various scenes to evaluate the performance and robustness of SIFNet. The results demonstrate the proposed SIFNet performed a comparable accuracy for template matching with other algorithms and was robust to geometric distortions and radiometric variations of multisource remote sensing data.

Keywords

• Multisource remote sensing registration
• Interaction fusion
• Transformer
• Template matching