A Robust Intelligent System for Text-Based Traffic Signs Detection and Recognition in Challenging Weather Conditions

Abstract

Traffic signs have great importance regarding smooth traffic flow and safe driving. However, due to many distractions and capricious factors, spotting and perceiving them may become hazardous. Traffic sign detection and recognition have gained popularity to put an end or to lessen the issue, and massive efforts have been realized in this regard. Despite considerable endeavors put together for traffic sign detection and recognition, there is a lack of attention in this area where these traffic signs contain text in them. A handful of studies may be found in state-of-the-art (SOTA) methods for text-based traffic sign detection, and particularly lesser for text recognition of detected text. The proposed method focuses on developing a robust semi-pipeline intelligent system to detect and understand text from traffic road signs boards in various weather conditions. For this purpose, a customized YOLOv5s is deployed for initial panel detection. Subsequently, MSER with preprocessing techniques is used for localization of text. Finally, OCR with NLP is utilized to recognize the text. The proposed method employed the ASAYAR dataset for training and different datasets for testing. The proposed approach produced satisfactory outcomes on them in contrast with SOTA approaches.

Keywords

• Text recognition
• deep learning
• natural language processing
• automated road signs/panels detection
• YOLOV5s
• MSER