Applied Sciences (Jun 2022)

Integration of Computer Vision and Convolutional Neural Networks in the System for Detection of Rail Track and Signals on the Railway

  • Aleksandar Dragan Petrović,
  • Milan Banić,
  • Miloš Simonović,
  • Dušan Stamenković,
  • Aleksandar Miltenović,
  • Gavrilo Adamović,
  • Damjan Rangelov

Journal volume & issue
Vol. 12, no. 12
p. 6045


Read online

One of the most challenging technical implementations of today is self-driving vehicles. An important segment of self-driving is the ability of the computer to “see/detect” objects of interest at a distance which enables safe vehicle operation. An algorithm for the detection of railway infrastructure objects, namely, track and signals, is proposed in this paper to enable detection of signals which are relevant for the track the train is moving along. The algorithm integrates traditional computer vision (CV) algorithms, including Canny edge detection, Hough transform, and You Only Look Once (YOLO) algorithm, based on convolutional neural networks (CNNs). Each of the concepts (CV and CNNs) deals with a different object of detection which together form a unique system that aims to detect both the rails and the relevant signals. This approach ensures that the artificial intelligence (AI) system is “aware” of which route the signal belongs to. The reliability of the proposed algorithm in detection of a relevant signal, verified by the performed tests, is up to 99.7%. The metric method used for validation was intersection over union (IoU). The obtained value of IoU applied on the entire validation dataset exceeds 0.7. Calculated values of average precision and recall were 0.89 and 0.76, respectively. The algorithm created in this way solves the problem of detection of relevant signals along the train route, especially in multitrack scenarios such as stations and yards.