3D sign language recognition using spatio temporal graph kernels

Abstract

3D Sign language recognition is challenging from capturing to recognition. 3D signs are a set of spatio temporal variations of hands and fingers with respect to face, head and torso. 3D motion capture technology has enabled us to capture these complex 3D human motions preserving 95% of the visual information required for recognition. A twin motion algorithm is proposed to recognize 3D signs with variable motion joints. Variable motions in joints arise duo to non-uniform distances between the joints. For example, finger motions are different from hand motions. A common measure to extract motion features from 3D skeletal data is relative range of joint relative distance (RRJRD). However, relative range of joint relative distance cannot quantify all the relative joint motions for characterizing a sign because of the different in motion ranges between different body parts used in defining a sign. Hence, we propose a wide RRJRD and narrow RRJRD based characterization to project the motion features on to graph. Each sign is characterized by a set of spatio temporal projections on to a constructed sign graph. The experiment results show that the proposed method is signer invariant, motion invariant and faster compared to state-of-the-art graph kernel methods.

Keywords

• 3D Sign language recognition
• 3D Joint relative distance
• Wide and Narrow motion Joints
• Restructured database model
• Graph kernel matching