Six-layer Optimized Convolutional Neural Network for Lip Language Identification

Abstract

INTRODUCTION: Lip language is one of the most important communication methods in social life for people with hearing impairment and impaired expression ability. This communication method relies on visual recognition to understand the meaning expressed in communication. OBJECTIVES: In order to improve the accuracy of this natural language recognition, we propose six-layer optimized convolutional neural network for lip recognition. METHODS: The calculation method of the convolutional layer in the CNN model is used, and two pooling methods are compared: the maximum pooling operation and the average pooling operation to analyse the most important feature data in the picture. In order to reduce the simulation in the model training process, the closing rate has been optimized by introducing Dropout technology. RESULTS: It shows that the recognition accuracy rate based on the six-layer convolutional neural network can reach 85.74% on average. This method can effectively recognize lip language. CONCLUSION: We propose a six-layer optimized convolutional neural network method for lip language recognition, and the identification of lip language features of this method is better than 3D+ DenseNet +1 × 1 Conv +resBi-LSTM, 3D+CNN, ConvNet+2 -256-LSTM+VGG-16 three advanced methods.

Keywords

• lip language identification
• convolutional neural network
• Batch Normalization
• dropout