Human‐action recognition using a multi‐layered fusion scheme of Kinect modalities

Abstract

This study addresses the problem of efficiently combining the joint, RGB and depth modalities of the Kinect sensor in order to recognise human actions. For this purpose, a multi‐layered fusion scheme concatenates different specific features, builds specialised local and global SVM models and then iteratively fuses their different scores. The authors essentially contribute in two levels: (i) they combine the performance of local descriptors with the strength of global bags‐of‐visual‐words representations. They are able then to generate improved local decisions that allow noisy frames handling. (ii) They also study the performance of multiple fusion schemes guided by different features concatenations, Fisher vectors representations concatenation and later iterative scores fusion. To prove the efficiency of their approach, they have evaluated their experiments on two challenging public datasets: CAD‐60 and CGC‐2014. Competitive results are obtained for both benchmarks.

Keywords

• human-action recognition
• multilayered fusion scheme
• Kinect modalities
• RGB
• depth modalities
• global SVM models