Non-Touch Character Input System Based on Hand Tapping Gestures Using Kinect Sensor

Abstract

There have been a lot of studies on the text input system using the image-based hand gesture recognition. However, hand gesture languages such as sign languages, finger alphabets, and aerial handwriting treated in the previous works have some problems to be commonly used. The aerial handwriting requires much time for writing and recognition. The sign languages and finger alphabets demand quite a knowledge and practice for using it, which results in restricting the number of their users. As a solution to the problems, this paper proposes a new character input system based on hand tapping gestures for Japanese hiragana and English characters that can be used to facilitate human-computer interaction. The hand tapping gestures are motions for tapping keys on aerial virtual keypads by hands, which can be effectively used as a hand alphabet by anyone, including hearing impaired individuals. For hiragana characters, the hand used for tapping a key and the number of stretched fingers of the hand decide the consonant part of characters, and thereby the aerial virtual keypad. The character to be entered is determined by tapping the key on the virtual keypad corresponding to the desired vowel. Because we adopt a key layout similar to the Japanese and English flick keyboard of smart phones, our hand tapping gestures can be easily used by anyone with only a brief description. The users can effectively interact with computers by using our non-touch input system where only the Kinect sensor is used without any keyboard, mouse or body-worn device. We expect that our character input system will open a new channel for human-computer interaction.

Keywords

• Non-touch character input
• hand gesture
• fingertip detection
• Kinect sensor