SIGNFORMER: DeepVision Transformer for Sign Language Recognition

Abstract

Sign language is the most common form of communication for the hearing impaired. To bridge the communication gap with such impaired people, a normal people should be able to recognize the signs. Therefore, it is necessary to introduce a sign language recognition system to assist such impaired people. This paper proposes the Transformer Encoder as a useful tool for sign language recognition. For the recognition of static Indian signs, the authors have implemented a vision transformer. To recognize static Indian sign language, proposed methodology archives noticeable performance over other state-of-the-art convolution architecture. The suggested methodology divides the sign into a series of positional embedding patches, which are then sent to a transformer block with four self-attention layers and a multilayer perceptron network. Experimental results show satisfactory identification of gestures under various augmentation methods. Moreover, the proposed approach only requires a very small number of training epochs to achieve 99.29 percent accuracy.

Keywords

• Transformer
• deep learning
• sign language
• multi head attention
• healthcare
• mental health