Flow‐assisted visual tracking using event cameras

Abstract

Abstract The data from event cameras not only portray contours of moving objects but also contain motion information inherently. Herein, motion information can be used in event‐based and frame‐based object trackers to ease the challenges of occluded objects and data association, respectively. In the event‐based tracker, events within a short interval are accumulated. Within the interval, the histogram of local time measurements (or ‘motion histogram’) is proposed as the feature to describe the target and candidate regions. Then the mean‐shift tracking approach is used by shifting the tracker towards similarity maximisation on motion histograms between target and candidate regions. As for the frame‐based tracker, given the assumption that a single object moves at a constant velocity on the image plane, the distribution of local timestamps is modelled, followed by which object‐level velocities are obtained from parameter estimation. We then build a Kalman‐based ensemble, in which object‐level velocities are deemed as an additional measurement on top of object detection results. Experiments have been conducted to measure the performance of proposed trackers based on our self‐collected data. Thanks to the assistance from motion information, the event‐based tracker successfully differentiates partially overlapped objects with distinct motion profiles; The inter‐frame tracker avoids data association failure on fast‐moving objects and leads to fast convergence on object velocity estimation.