Super-Resolution of Sentinel-2 RGB Images with VENµS Reference Images Using SRResNet CNNs

Abstract

Super-resolution (SR) is a well-established technique used to enhance the resolution of low-resolution images. In this paper, we introduce a novel approach for the super-resolution of Sentinel-2 10 m RGB images using higher-resolution Venus 5 m RGB images. The proposed method takes advantage of a modified SRResNet network, integrates perceptual loss based on the VGG network, and incorporates a learning rate decay strategy for improved performance. By leveraging higher-resolution VENµS 5 m RGB images as reference images, this approach aims to generate high-quality super-resolved images of Sentinel-2 10 m RGB images. The modified SRResNet network was designed to capture and learn underlying patterns and details present in Venus images, enabling it to effectively enhance the resolution of Sentinel-2 images. In addition, the inclusion of perceptual loss based on the VGG network helps preserve important visual features and maintain the overall image quality. The learning rate decay strategy ensures the network converges to an optimal solution by gradually reducing the learning rate during the training process. Our research contributes to the field of super-resolution by offering a novel approach specifically tailored for enhancing the resolution of Sentinel-2 10 m RGB images using Venus 5 m RGB images. The proposed methodology has the potential to benefit various applications, such as remote sensing, land cover analysis, and environmental monitoring, where high-resolution imagery is crucial for accurate and detailed analysis. In summary, our approach presents a promising solution for the super-resolution of Sentinel-2 10 m RGB images, providing an effective means to obtain higher-resolution imagery by leveraging the complementary information from Venus 5 m RGB images. We used the SEN2VENµS dataset for this research. The SEN2VENµS dataset comprises cloud-free surface reflectance patches obtained from Sentinel-2 imagery. Notably, these patches are accompanied by corresponding reference surface reflectance patches captured at a remarkable 5 m resolution by the VENµS Micro-Satellite on the same acquisition day. To assess the effectiveness of the proposed approach, we evaluated it using widely used metrics such as the mean squared error (MSE), the peak signal-to-noise ratio (PSNR), and the structural similarity index (SSIM). These metrics provided quantitative measurements of the quality and fidelity of the super-resolved images. Experimental results demonstrate the effectiveness of our proposed approach in achieving improved super-resolution performance compared to existing methods. As an example, our method achieved a PSNR of 35.70 and a SSIM of 0.94 on the training dataset, outperforming the bicubic interpolation method, which yielded a PSNR of 29.53 and a SSIM of 0.92. On the validation dataset, our approach achieved a PSNR of 40.3809 and a SSIM of 0.98, while the bicubic interpolation method achieved a PSNR of 34.26 and a SSIM of 0.94. Finally, on the test dataset, our approach achieved a PSNR of 29.8231 and a SSIM of 0.90, whereas the bicubic interpolation method yielded a PSNR of 26.99 and a SSIM of 0.85. The evaluation based on MSE, PSNR, and SSIM metrics showcases the enhanced visual quality, increased image resolution, and improved similarity to the reference Venus images.

Keywords

• super-resolution
• remote sensing
• Sentinel-2
• deep learning
• SRResNet
• perceptual loss