SIGNSENSEI: ADVANCING UNIVERSAL COMMUNICATION THROUGH DEEP LEARNING

Abstract

One of the most effective systems and tools for communication in case of deaf people is Sign Language.This paper proposed a system called ‘SignSensei’ which creates a platform for deaf people to converse using sign language regardless of whether they are familiar with sign language or not. SignSensei is a web application that uses computer vision and machine learning to translate and interpret hand gestures that are recorded using a webcam. It uses Tensorflow.JS and the Deeplearn.js module, and uses the k-Nearest Neighbours (kNN) method to classify images. This paper explores the comprehensive development, implementation, and impact of SignSensei. SignSensei aims to empower users by allowing them to train the system with their own words on the website. This paper delves into the unique features of SignSensei, including its user-generated word training and the integration of speech recognition capabilities, enabling both sign and speech communication.

Keywords

• deep learning
• sign language recognition
• real time image analysis
• tensorflow.js