Object Detection in Thermal Spectrum for Advanced Driver-Assistance Systems (ADAS)

Abstract

AI-based smart thermal perception systems can cater to the limitations of conventional imaging sensors by providing a more reliable data source in low-lighting conditions and adverse weather conditions. This research evaluates and modifies the state-of-the-art object detection and classifier framework for thermal vision with seven key object classes in order to provide superior thermal sensing and scene understanding input for advanced driver-assistance systems (ADAS). The networks are trained on public datasets and is validated on test data with three different test approaches which include test-time augmentation, test-time with no augmentation, and test-time with model ensembling. Additionally, a new model ensemble-based inference engine is proposed, and its efficacy is tested on locally gathered novel test data comprising of 20K thermal frames captured with an uncooled LWIR prototype thermal camera in challenging weather and environmental scenarios. The performance analysis of trained models is investigated by computing precision, recall, and mean average precision scores (mAP). Furthermore, the smaller network variant of thermal-YOLO architecture is optimized using TensorRT inference accelerator, which is then deployed on GPU and resource-constrained edge hardware Nvidia Jetson Nano. This is implemented to explicitly reduce the inference time on GPU as well as on Nvidia Jetson Nano to evaluate the feasibility for added real-time onboard installations.

Keywords

• Thermal-infrared
• object detection
• advanced driver-assistance systems
• deep learning
• edge computing