Object Detection Using ESRGAN With a Sequential Transfer Learning on Remote Sensing Embedded Systems

Abstract

The field of remote sensing has experienced rapid advancement owing to the widespread utilization of image sensors, drones, and satellites for data collection. However, object detection in remote sensing poses challenges owing to small objects with low resolution (LR), complex scenes, and limited data for model training. Conventional methods rely on computationally intensive models and hardware setups that are not suitable for real-time detection. To address this issue, we propose a novel sequential transfer learning method based on generative adversarial networks (GANs) that generate super-resolved data from LR for embedded systems, enabling improved performance with limited data by combining learning from both heterogeneous and homogeneous data. Additionally, we train the model sequentially, starting with the easiest data and progressing to the most complex based on the complexity levels determined by the GAN-generated images. The GAN model is trained on a diverse dataset of images and learned to generate high-resolution images from the LR, capturing finer object details for enhanced accuracy and localization capabilities. The proposed method acquires more robust features and enhances the generalizability and convergence of the model. Furthermore, the trained model of the proposed method is deployed on embedded platforms, such as Nvidia’s Jetson Nano and AGX Orin, for real-time remote-sensing object detection, with satisfactory detection performance. Evaluation metrics, such as [email protected], [email protected]–0.95, and F1 score were used to assess the object detection accuracy. The experimental results demonstrated a significant improvement in accuracy when the proposed method was implemented with YOLOv7, achieving detection performance scores of 99.21, 98.57, 93.71, 78.38, 75.73, 48.68, 0.971, 0.971, and 0.911 on the VEDAI-VISIBLE, VEDAI-IR, and DOTA datasets, respectively.

Keywords

• Embedded system
• Jetson AGX Orin
• Jetson nano
• real-time detection
• remote sensing images
• super-resolution