Surveying You Only Look Once (YOLO) Multispectral Object Detection Advancements, Applications, and Challenges

Abstract

Multispectral imaging and deep learning have emerged as powerful tools supporting diverse use cases from autonomous vehicles to agriculture, infrastructure monitoring and environmental assessment. The combination of these technologies has led to significant advancements in object detection, classification, and segmentation tasks in the non-visible light spectrum. This paper considers 400 total papers, reviewing 200 in detail to provide an authoritative meta-review of multispectral imaging technologies, deep learning models, and their applications, considering the evolution and adaptation of you only look once (YOLO). Ground-based collection is the most prevalent approach, totaling 63% of the papers reviewed, although uncrewed aerial systems (UAS) for YOLO-multispectral applications have doubled since 2020. The most prevalent sensor fusion is red-green-blue (RGB) with long-wave infrared (LWIR), comprising 39% of the literature. YOLOv5 remains the most used variant for adaption to multispectral applications, consisting of 33% of all modified YOLO models reviewed. Future research needs to focus on: 1) developing adaptive YOLO architectures capable of handling diverse spectral inputs that do not require extensive architectural modifications; 2) exploring methods to generate large synthetic multispectral datasets; 3) advancing multispectral YOLO transfer learning techniques to address dataset scarcity; and 4) innovating fusion research with other sensor types beyond RGB and LWIR.

Keywords

• Multispectral object detection
• you only look once (YOLO)
• convolutional neural networks (CNN)
• deep learning
• RGB
• LWIR