DCDGAN-STF: A Multiscale Deformable Convolution Distillation GAN for Remote Sensing Image Spatiotemporal Fusion

Abstract

Remote sensing image spatiotemporal fusion (STF) aims to generate composite images with high-temporal and spatial resolutions by combining remote sensing images captured at different times and with different spatial resolutions (DTDS). Among the existing fusion algorithms, deep learning-based fusion models have demonstrated outstanding performance. These models treat STF as an image super-resolution problem based on multiple reference images. However, compared to traditional image super-resolution tasks, remote sensing image STF involves merging a larger amount of multitemporal data with greater resolution difference. To enhance the robust matching performance of spatiotemporal transformations between multiple sets of remote sensing images captured at DTDS and to generate super-resolution composite images, we propose a feature fusion network called the multiscale deformable convolution distillation generative adversarial network (DCDGAN-STF). Specifically, to address the differences in multitemporal data, we introduce a pyramid cascading deformable encoder to identify disparities in multitemporal images. In addition, to address the differences in spatial resolution, we propose a teacher–student correlation distillation method. This method uses the texture details' disparities between high-resolution multitemporal images to guide the extraction of disparities in blurred low-resolution multitemporal images. We comprehensively compared the proposed DCDGAN-STF with some state-of-the-art algorithms on two landsat and moderate-resolution imaging spectroradiometer datasets. Ablation experiments were also conducted to test the effectiveness of different submodules within DCDGAN-STF. The experimental results and ablation analysis demonstrate that our algorithm achieves superior performance compared to other algorithms.

Keywords

• Deformable convolution
• generative adversarial network (GAN)
• remote sensing image spatiotemporal fusion (STF)
• teacher–student correlation distillation