YOLOFIV: Object Detection Algorithm for Around-the-Clock Aerial Remote Sensing Images by Fusing Infrared and Visible Features

Abstract

With the rapid advancements in deep learning technology, various deep learning-based object detection algorithms have found extensive applications in UAV-related tasks. However, motivated by the fact that current object detection algorithms for unimodal aerial remote sensing images fail to achieve around-the-clock object detection. To tackle this, we propose an around-the-clock object detection algorithm YOLOFIV that fuses infrared and visible features. First, we design a dual-stream backbone network based on the attention mechanism to adequately extract the features of both modalities. Moreover, the ECA attention mechanism is integrated into the feature enhancement network to amplify attention toward challenging detection scenarios. Finally, we improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method YOLOFIV on the widely used drone vehicle dataset, YOLOFIV achieves an accuracy of 64.71% (in terms of $\text{mean average precision}_{0.5}$), accuracy improvement of 8.32% over baseline bimodal model, similar performance to UACMD designed for ARSI object detection but with 92.35% reduction in parameter count, and 17.87 times speedup. The results show that our approach achieves round-the-clock object detection while maintaining a favorable accuracy-speed tradeoff.

Keywords

• Attention mechanism
• feature fusion
• multimodal aerial remote sensing images (ARSIs)
• object detection
• round-the-clock detection