RF-CSign: A Chinese Sign Language Recognition System Based on Large Kernel Convolution and Normalization-Based Attention

Abstract

Hearing impaired people use sign language for communication, which relies on the movement gestures of body parts and plays a vital role in human-computer interaction. Most wireless sensing-based gesture recognition studies have recognized simple gestures but overlooked the recognition of complex activities, such as sign language. In addition, cross-domain recognition often requires a large amount of data to train classifiers for each environment. Therefore, we propose RF-CSign, which aims to achieve high accuracy in sign language recognition and cross-domain recognition. First, we use Radio Frequency Identification (RFID) to collect signals and obtain denoised signals through data pre-processing, so that they can be processed in a neural network. Second, the RF-CSign network is proposed with the inclusion of large kernel convolution to reduce the complexity of the model and to make the model with long-range correlations, thereby enhancing recognition accuracy. Third, RF-CSign employs a pixel Normalization-based Attention Module (NAM) to enhance the stability of the model, thereby addressing the problem of model overfitting. Finally, RF-CSign achieves high accuracy in cross-domain environments through a migration learning approach. The experimental results showed that the average recognition accuracy of RF-CSign reached 99.17%, and the average recognition accuracy for new users and new environments recorded 96.67% and 97.50%, respectively.

Keywords

• Sign language recognition
• RFID
• large kernel convolution
• transfer learning