AMERICAN SIGN LANGUAGE FINGERSPELLING USING HYBRID DISCRETE WAVELET TRANSFORM-GABOR FILTER AND CONVOLUTIONAL NEURAL NETWORK

Abstract

American Sign Language (ASL) is widely used for communication by deaf and mute people. In fingerspelling, the letters of the writing system are represented using only hands. Generally, hearing people do not understand sign language and this creates a communication gap between the signer and speaker community. A real-time ASL fingerspelling recognizer can be developed to solve this problem. Sign language recognizer can also be trained for other applications such as human-computer interaction. In this paper, a hybrid Discrete Wavelet TransformGabor filter is used on the colour images to extract features. Classifiers are evaluated on signer dependent and independent datasets. For evaluation, it is very important to consider signer dependency. Random Forest, Support Vector Machine and K-Nearest Neighbors classifiers are evaluated on the extracted set of features to classify the 24 classes of ASL alphabets with 95.8%, 94.3% and 96.7% accuracy respectively on signer dependent dataset and 49.16%, 48.75% and 50.83% accuracy respectively on signer independent dataset. Lastly, Convolutional Neural Network was also trained and evaluated on both, which produced 97.01% accuracy on signer dependent and 76.25% accuracy on signer independent dataset.

Keywords

• ASL
• CNN
• DWT
• Gabor filter
• Histogram of distances from COM