Crowd counting by the dual‐branch scale‐aware network with ranking loss constraints

Abstract

Image crowd counting is a challenging problem. This study proposes a new deep learning method that estimates crowd counting for the congested scene. The proposed network is composed of two major components: the first ten layers of VGG16 are used as the backbone network, and a dual‐branch (named as Branch_S and Branch_D) network is proposed to be the second part of the network. Branch_S extracts low‐level information (head blob) through a shallow fully convolutional network and Branch_D uses a deep fully convolutional network to extract high‐level context features (faces and body). Features learnt from the two different branches can handle the problem of scale variation due to perspective effects and image size differences. Features of different scales extracted from the two branches are fused to generate predicted density map. On the basis of the fact that an original graph must contain more or equal number of persons than any of its sub‐images, a ranking loss function utilising the constraint relationship inside an image is proposed. Moreover, the ranking loss is combined with Euclidean loss as the final loss function. Our approach is evaluated on three benchmark datasets, and better results are achieved compared with the state‐of‐the‐art works.

Keywords

• deep learning method
• congested scene
• VGG16
• Branch_S
• Branch_D
• shallow fully convolutional network