Integration of gradient guidance and edge enhancement into super‐resolution for small object detection in aerial images

Abstract

Abstract Detecting small objects are difficult because of their poor‐quality appearance and small size, and such issues are especially pronounced for aerial images of great importance. To address the small object detection (SOD) problem, a united architecture that tries to upsample small objects into super‐resolved versions, achieving characteristics similar to those large objects and thus resulting in more discriminative detection is used. For this purpose, a new end‐to‐end multi‐task generative adversarial network (GAN) is proposed. In the architecture, the generator is a super‐resolution (SR) network, and the discriminator is a multi‐task network. In the generator, a gradient guide and an edge‐enhancement strategy are introduced to alleviate structural distortions. In the discriminator, a faster region‐based convolutional neural network (FRCNN) is incorporated for the task of object detection. Specifically, the discriminator outputs a distribution scalar to measure the realness. Then, each super‐resolved image passes through the discriminator with a realness distribution, classification scores, and bounding box regression offsets. Furthermore, the losses of the detection task are backpropagated into the generator during training rather than being optimized independently. Extensive experiments on the challenging cars overhead with context dataset (COWC), detectIon in optical remote sensing images (DIOR), vision meets drones (VisDrone), and dataset for object detection in aerial images (DOTA) demonstrate the effectiveness of the proposed method in reconstructing structures while generating natural super‐resolved images and show the superiority of the proposed method in detecting small objects over state‐of‐the‐art detectors.