IEEE Transactions on Neural Systems and Rehabilitation Engineering (Jan 2022)

Touchless Head-Control (THC): Head Gesture Recognition for Cursor and Orientation Control

  • Wahyu Rahmaniar,
  • Alfian Ma'Arif,
  • Ting-Lan Lin

DOI
https://doi.org/10.1109/TNSRE.2022.3187472
Journal volume & issue
Vol. 30
pp. 1817 – 1828

Abstract

Read online

The touchless techniques in human-computer interaction (HCI) can effectively expand computer access capabilities for disabled people. This paper presents Touchless Head-Control (THC), an assistive system method for computer cursor control based on head pose captured with an RGB camera. Our work aimed to replace the standard cursor control using a device on the user’s head. The convolutional neural networks with predicted fine-grained feature maps and binned classification were applied to estimate the head pose angles. The mouse pointer or cursor is moved to actual locations on the screen based on head movement (yaw and pitch) and the center position of the face. Head tilt to the right or left (roll) to control the mouse button. In addition, the proposed method can be used to simulate the movement of the robot or joystick using the head to control objects within three degrees of freedom (DOF). Various participants were involved in the interaction design evaluation, in which target selection accuracy, travel time, and path efficiency were measured. This technology allows people with limited motor skills to easily control a PC cursor and 3D object orientation without the use of additional equipment or sensors.

Keywords