SDDT-SR: A Scale-Decoupling Super-Resolution Network With Domain Transfer for Heterogeneous Images

Abstract

Super-resolution image reconstruction (SRR) is an important task in remote sensing image (RSI) processing, which can serve the application fields, such as natural resources exploration and agricultural monitoring. Recently deep learning based methods achieved great success in RSI-SRR, making RSI-SRR more efficient and accurate. However, homogeneous low-resolution (LR) and high-resolution RSI pairs from the same sensor are often difficult to obtain, and the use of simulated image pairs makes it difficult to apply these methods in real scenarios. Some RSI-SRR methods attempt to reconstruct image resolution from heterogeneous RSIs. But these methods, on the one hand, are difficult to deal with large-scale RSI-SRR; and on the other hand, they do not take into account the domain differences of heterogeneous RSIs. To address the above challenges, this article proposes a scale-decoupling super-resolution network with domain transfer for heterogeneous RSIs (SDDT-SR). The SDDT-SR breaks down the large-scale SSR (x8) into two cascaded smaller-scale steps (x2, x4), progressively restoring spatial details of the images. Specifically, in SDDT-SR, an SR module with domain transfer (DTSR) is designed to transfer the LR image from a specific domain to another one, and an SR module with unpaired learning (ULSR) is designed to further up-sample the output of DTSR to the target resolution. Experiments on SGSRD and satellite-UAV heterogeneous image change detection dataset (HTCD) dataset show that SDDT-SR outperforms existing methods in both metrics and visualization, with the fréchet inception distance (FID) of 181.8679 on SGSRD and of 163.3831 on HTCD.

Keywords

• Deep learning (DL)
• heterogeneous images
• remote sensing
• super-resolution
• unpaired dataset