Semantic‐aware spatial regularization correlation filter for visual tracking

Abstract

Abstract Correlation filters with convolutional neural network (CNN) features have been successfully applied to visual tracking owing to their impressive combined capability for object representation. Unfortunately, further performance improvement is limited due to unwanted boundary effects of the circular structure. In this work, through an in‐depth study of the features’ characteristics, the authors propose a novel tracking strategy to achieve simultaneous filter matching and regularization with CNN features when tracking is on the fly. With a feature decomposed transform matrix, a spatial semantic regularization is generated to reduce the boundary effect effectively during filter optimization. Before each output, the regularized filter is then back performed to match with the extracted features of a search region to find the optimum candidate. Specifically, the most important advantage of the proposed spatial semantic map is to initialize only in the first frame as all the other tracking strategies. Besides, the authors design a novel updating strategy to tackle the cases where the object is occluded or disappeared in the scene. At this time, the maximum of the map is small, even negative. A substantial experiment has been carried out on the popular benchmark tracking datasets; the reliable results have demonstrated that the authors’ method is able to outperform most of the state‐of‐the‐art tracking works in both accuracy and robustness.