Quantitative Analysis on the Interaction Fatigue of Natural Gestures

Abstract

With the popularity of natural user interface (NUI), natural gesture interaction has become the mainstream. Using improper natural gestures for a long time will cause muscle fatigue, which leads to an increase in pathological problems such as tenosynovitis. To avoid the harm caused by improper gestures, this paper selects three daily interactive gestures as the research objects including browsing information, playing games and typing, and divides them into nine independent gestures. After denoising, filtering, segmenting, and extracting the parameters of the acquired surface electromyography (sEMG) signals, the time-domain, frequency-domain and time-frequency-domain are analyzed. The characteristics of the envelope waveform and power spectrum threshold, as well as the fatigue characteristics of nine independent gestures are obtained. The long short-term memory (LSTM), one of the recurrent neural network (RNN) methods is used to train nine independent gesture models. The fatigue characteristics of the integrated gestures are predicted by the trained LSTM series model. The energy consumption characteristics of integrated gestures in smartphones and PCs are obtained. It is found that the simple behavior of browsing in the integrated behavior is suitable for natural interactive gestures, and complex behaviors such as games or typing on PC have lower energy consumption than that of smartphones. Among the independent natural gestures, the energy consumption of clicking is higher than that of dragging. Compared with a behavior for the same purpose, the energy consumption of mouse gestures for PCs is much lower than that of smartphones. This study of gesture fatigue provides a theoretical basis for the design of gestures and the development of internet user products.

Keywords

• Fatigue analysis
• long short-term memory (LSTM)
• gestures design
• surface electromyography (sEMG)