A Real-Time Intelligent System Based on Machine-Learning Methods for Improving Communication in Sign Language

Abstract

In this article, we introduce a cost-effective and real-time intelligent system tailored to Pakistan sign language (PSL) recognition, aimed at facilitating communication for hearing-impaired individuals. The system utilizes a specialized glove equipped with flex sensors and an MPU-6050 device to capture finger movements and hand orientation in a three-dimensional space. A dataset comprising ten unique PSL signs, each performed by five participants for a total of 5000 samples, was used to train machine learning classifiers. These signs involve single-hand and single-movement gestures, optimizing the system for real-time PSL recognition. Machine learning classifiers, including decision trees, k-nearest neighbors, and support vector machines, achieved accuracy levels of 96%, 96.5%, and 97%, respectively. While direct quantitative comparisons with state-of-the-art systems are limited due to the uniqueness of PSL, we discuss our system in the context of recent advancements in sign language recognition. Real-time testing underscores the system’s practical applicability and portability, demonstrating its potential for deployment in resource-constrained settings as an accessible initial step toward more comprehensive PSL recognition solutions.

Keywords

• Assistive technology
• flex sensor glove
• Kalman filter
• MPU-6050 device
• Raspberry Pi 3B microcontroller
• real-time processing