Frequency Domain-Based Super Resolution Using Two-Dimensional Structure Consistency for Ultra-High-Resolution Display

Abstract

Recent advancements in the field of super resolution (SR) have seen the adoption of generative adversarial networks (GANs) for realistic images. In this case, when performing with low-resolution (LR) images, several challenges arise due to the loss of high-frequency details from high-resolution (HR) images, potentially leading to unwanted distortions in the generated SR images. Our paper presents a novel solution by using two-dimensional structure consistency (TSC) for image analysis. The TSC serves as a mask, enabling adaptive analysis based on the unique frequency characteristics of different image regions. Furthermore, a mutual loss mechanism, which dynamically adjusts the training process based on the results filtered by the TSC-based mask, is introduced. Additionally, the TSC loss is proposed to enhance our model capacity to generate precise TSC in high-frequency regions. As a result, our method effectively reduces distortions in high-frequency areas while preserving clarity in regions containing low-frequency components. Our method outperforms other SR techniques, demonstrating superior results in both qualitative and quantitative evaluations. Quantitative measurements, including PSNR, SSIM, and the perceptual metric LPIPS, show comparable PSNR and SSIM values, while the perceptual SR quality is notably improved according to the LPIPS metric.

Keywords

• interpolation
• image up-scaling
• super resolution
• deep learning