Gong-kuang zidonghua (Nov 2023)
Bimodal environment perception technology for underground coal mine based on radar and visual fusion
Abstract
Environmental perception is a key technology for scenario applications such as coal mine inspection robots and visual measurement systems. The single modal environmental perception technology has poor perception capability for complex environments in underground coal mines. A bimodal space fusion method for radar and vision has been proposed. The modal achieves the fusion of information collected by LiDAR and camera through coordinate conversion, thereby improving environmental perception capability. In order to better extract object feature information, a bimodal fusion environment perception network architecture technology route is proposed. The environmental information collected by the camera and radar is fused and processed by the radar and visual bimodal space fusion method. The multimodal feature fusion network module extracts object features from the fused information. The multitask processing network module uses different task heads to process object feature information, completing environmental perception tasks such as object detection, image segmentation, and object classification. The experiment is conducted using the YOLOv5s object detection algorithm to build a bimodal feature extraction network module. The results show that the success rate of the bimodal environment perception technology for underground coal mine based on radar and visual fusion for personnel detection in underground roadway environments is improved by 15% and 10% compared to visual and radar perception, respectively. The mean average precision of segmentation for various types of objects such as lane lines and signs are improved by more than 10% compared to visual perception. It effectively improves the perception capability of underground environment in coal mines, providing technical support for application scenarios such as coal mine road environment perception, visual measurement systems, unmanned mining vehicle navigation systems, and mine search and rescue robots.
Keywords
