Effective Fusion Method on Silhouette and Pose for Gait Recognition

Abstract

Silhouette and pose are two common features to extract the descriptive and unique patterns of a person’s gait, and good performance has been already achieved driven by the deep learning techniques. However, some issues still exist, the silhouette is known to be sensitive to the changes of the appearance while pose is not so discriminative as silhouette even though it is considered as being more robust. Therefore, it is advantageous to fuse the two features into one model to achieve both the accuracy as well as the robustnesss. In this paper, we propose a simple yet effective fusion model to combine both the features, where the two features are first scaled by normalisation and then combined by the Compact Bilinear Pooling to model the higher order and fine-grained information. The superiority of the proposed method is verified through experiments on benchmark datasets CASIA-B, OUMVLP, and SOTON-small. In CASIA-B, we achieved SOTA results with an average of 96.9% rank-1 accuracy. In addition, cross data experiments are conducted to demonstrate the robustness of our method.

Keywords

• Gait recognition
• pattern recognition
• multi-modal
• deep learning