Hybrid InceptionNet Based Enhanced Architecture for Isolated Sign Language Recognition

Abstract

Sign language is a common way of communication for people with hearing and/or speaking impairments. AI-based automatic systems for sign language recognition are very desirable since they can reduce barriers between people and improve Human-Computer Interaction (HCI) for the impaired community. Automatically recognizing sign language is still an open challenge since the sign language itself has a complex structure to convey messages. The key role is played by the isolated signs that refer to single gestures carried out by hand movements. In the last decade, research has improved the automatic recognition of isolated sign language from videos using machine learning approaches. Starting from a comprehensive analysis of existing recognition techniques, with an in-depth focus on existing public datasets, the study proposes an advanced convolution-based hybrid Inception architecture to improve the recognition accuracy of isolated signs. The main contributions are to enhance InceptionV4 with optimized backpropagation through uniform connections. Besides, an ensemble learning framework with different Convolution Neural Networks has been also introduced and exploited to further increase the recognition accuracy and robustness of isolated sign language recognition systems. The effectiveness of the proposed learning approaches has been proved on a benchmark dataset of isolated sign language gestures. The experimental results demonstrate that the proposed ensemble model outperforms sign identification, yielding higher recognition accuracy (98.46%) and improved robustness.

Keywords

• Sign language recognition
• gesture recognition
• isolated sign
• deep learning
• computer vision