Hybrid Target Selections by ”Hand Gestures + Facial Expression” for a Rehabilitation Robot
Yi Han,
Xiangliang Zhang,
Ning Zhang,
Shuguang Meng,
Tao Liu,
Shuoyu Wang,
Min Pan,
Xiufeng Zhang,
Jingang Yi
Affiliations
Yi Han
The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China
Xiangliang Zhang
The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China
Ning Zhang
Key Laboratory of Rehabilitation Technical Aids Technology and System of the Ministry of Civil Affairs, National Research Center for Rehabilitation Technical Aids, Beijing 100176, China
Shuguang Meng
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650000, China
Tao Liu
The State Key Laboratory of Fluid Power and Mechatronic Systems, School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China
Shuoyu Wang
Department of Intelligent Mechanical Systems Engineering, Kochi University of Technology, 185 Miyanokuchi, Tosayamada-Cho, Kochi 782-8502, Japan
Min Pan
Department of Mechanical Engineering, University of Bath, Bath BA2 7AY, UK
Xiufeng Zhang
Key Laboratory of Rehabilitation Technical Aids Technology and System of the Ministry of Civil Affairs, National Research Center for Rehabilitation Technical Aids, Beijing 100176, China
Jingang Yi
Department of Mechanical and Aerospace Engineering, Rutgers University, Piscataway, NJ 08854, USA
In this study we propose a “hand gesture + face expression” human machine interaction technique, and apply this technique to bedridden rehabilitation robot. “Hand gesture + Facial expression” interactive technology combines the input mode of gesture and facial expression perception. It involves seven basic facial expressions that can be used to determine a target selecting task, while hand gestures are used to control a cursor’s location. A controlled experiment was designed and conducted to evaluate the effectiveness of the proposed hybrid technology. A series of target selecting tasks with different target widths and layouts were designed to examine the recognition accuracy of hybrid control gestures. An interactive experiment applied to a rehabilitation robot is designed to verify the feasibility of this interactive technology applied to rehabilitation robots. The experimental results show that the “hand + facial expression” interactive gesture has strong robustness, which can provide a novel guideline for designing applications in VR interfaces, and it can be applied to the rehabilitation robots.
