Sec-CLOCs: Multimodal Back-End Fusion-Based Object Detection Algorithm in Snowy Scenes

Abstract

LiDAR and cameras, often regarded as the “eyes” of intelligent driving vehicles, are vulnerable to adverse weather conditions like haze, rain, and snow, compromising driving safety. In order to solve this problem and enhance the environmental sensing capability under severe weather conditions, this paper proposes a multimodal back-end fusion object detection method, Sec-CLOCs, which is specifically optimized for vehicle detection under heavy snow. This method achieves object detection by integrating an improved YOLOv8s 2D detector with a SECOND 3D detector. First, the quality of image data is enhanced through the Two-stage Knowledge Learning and Multi-contrastive Regularization (TKLMR) image processing algorithm. Additionally, the DyHead detection head and Wise-IOU loss function are introduced to optimize YOLOv8s and improve 2D detection performance.The LIDROR algorithm preprocesses point cloud data for the SECOND detector, yielding 3D object detection results. The CLOCs back-end fusion algorithm is then employed to merge the 2D and 3D detection outcomes, thereby enhancing overall object detection capabilities. The experimental results show that the Sec-CLOCs algorithm achieves a vehicle detection accuracy of 82.34% in moderate mode (30–100 m) and 81.76% in hard mode (more than 100 m) under heavy snowfall, which demonstrates the algorithm’s high detection performance and robustness.

Keywords

• DyHead
• LIDROR
• multimodal object detection
• Sec-CLOCs
• Wise-IOU
• YOLOv8s