Embedded Descriptor Generation in Faster R-CNN for Multi-Object Tracking

Abstract

With the rapid growth of computer usage to extract the required knowledge from a huge amount of information, such as a video file, significant attention has been brought towards multi-object detection and tracking. Artificial Neural Networks (ANNs) have shown outstanding performance in multi-object detection, especially the Faster R-CNN network. In this study, a new method is proposed for multi-object tracking based on descriptors generated by a neural network that is embedded in the Faster R-CNN. This embedding allows the proposed method to directly output a descriptor for each object detected by the Faster R-CNN, based on the features detected by the Faster R-CNN to detect the object. The use of these features allows the proposed method to output accurate values rapidly, as these features are already computed for the detection and have been able to provide outstanding performance in the detection stage. The descriptors that are collected from the proposed method are then clustered into a number of clusters equal to the number of objects detected in the first frame of the video. Then, for further frames, the number of clusters is increased until the distance between the centroid of the newly created cluster and the nearest centroid is less than the average distance among the centroids. Newly added clusters are considered for new objects, whereas older ones are kept in case the object reappears in the video. The proposed method is evaluated using the UA-DETRAC (University at Albany Detection and Tracking) dataset and has been able to achieve 64.8% MOTA and 83.6% MOTP, with a processing speed of 127.3 frames per second.

Keywords

• convolutional neural networks
• multi-object detection
• multi-object tracking