Early estimation model for 3D-discrete indian sign language recognition using graph matching

Abstract

Machine translation of sign language is a critical task of computer vision. In this work, we propose to use 3D motion capture technology for sign capture and graph matching for sign recognition. Two problems related to 3D sign matching are addressed in this work: (1) how to identify same signs with different number of motion frames and (2) sign extraction from a clutter of non-sign hand motions. These two problems make the 2D or 3D sign language machine translation a challenging task. We propose graph matching with early estimation model to address these problems in two phases. The first phase consists of intra graph matching for motion frame extraction, which retains motion intensive frames in database and query 3D videos. The second phase applies inter graph matching with early estimation model on motion extracted query and dataset 3D videos. The proposed model increases the speed of the graph matching algorithm in estimating a sign with fewer frames. To test the graph matching model, we recorded 350 words of Indian sign language with 3D motion capture technology. For testing 4 variations per sign are captured for all signs with 5 different signers at same, slower, faster hand speeds and sign mixed cluttered hand motions. The early estimation graph matching model is tested for accuracy and efficiency in classifying 3D signs with the two induced real time constraints. In addition to 3D sign language dataset, the proposed method is validated on five benchmark datasets and against the state-of-the-art graph matching methods.

Keywords

• 3D motion capture
• 3D sign language
• 3D graph matching
• Pattern classification
• Spatial graph matching
• Temporal graph matching