Hand Trajectory Recognition by Radar with a Finite-State Machine and a Bi-LSTM

Abstract

Gesture plays an important role in human–machine interaction. However, the insufficient accuracy and high complexity of gesture recognition have blocked its widespread application. A gesture recognition method that combines state machine and bidirectional long short-term memory (Bi-LSTM) fusion neural network is proposed to improve the accuracy and efficiency. Firstly, gestures with large movements are categorized into simple trajectory gestures and complex trajectory gestures in advance. Afterwards, different recognition methods are applied for the two categories of gestures, and the final result of gesture recognition is obtained by combining the outputs of the two methods. The specific method used is a state machine that recognizes six simple trajectory gestures and a bidirectional LSTM fusion neural network that recognizes four complex trajectory gestures. Finally, the experimental results show that the proposed simple trajectory gesture recognition method has an average accuracy of 99.58%, and the bidirectional LSTM fusion neural network has an average accuracy of 99.47%, which can efficiently and accurately recognize 10 gestures with large movements. In addition, by collecting more gesture data from untrained participants, it was verified that the proposed neural network has good generalization performance and can adapt to the various operating habits of different users.

Keywords

• deep learning
• hand gesture recognition
• millimeter-wave radar