Recognizing multiclass Static Sign Language words for deaf and dumb people of Bangladesh based on transfer learning techniques

Abstract

Sign language is a language used for communication of the deaf and dumb (D&D) community. To avoid difficulties in communication among themselves and also among normal people, transfer learning-based automatic sign language recognition can play a great role. Transfer learning has been used in many countries in sign language. In Bangladesh, here also some techniques consisting of a convolutional neural network and transfer learning have been used to recognize Bangladeshi Sign language. Those techniques have been used only to sign alphabets, characters, and numbers. In this paper, a transfer learning based automatic sign language recognition system is introduced using Bangladeshi Sign Language (BdSL) words. Very rare research has been done on Bangladeshi Sign Words, and there is an inadequate Bangladeshi Sign Words dataset. This system employs four well-performed transfer learning techniques named VGG16, VGG19, AlexNet and InceptionV3 with pre-trained weights. The accuracy, recall, precision, and F1 score are used to assess the efficiency of the suggested models. The dataset of Bangladeshi sign words has been used in this paper which is consisting of 1105 images. The models show the training accuracy of 99.92%, 99.58%, 98.70% and 97.86% for VGG16, VGG19, InceptionV3 and AlexNet respectively whereas validation accuracy is 92.41%, 91.62%, 88.22% and 84.95% for VGG16, VGG19, InceptionV3 and AlexNet respectively. The proposed transfer learning based on the CNN method demonstrates better performance for the recognition of Bangladeshi Sign Words.

Keywords

• Transfer learning
• VGG16
• VGG19
• InceptionV3
• AlexNet
• Bengali Sign Word