Smart Control of Home Appliances Using Hand Gesture Recognition in an IoT-Enabled System

Abstract

Recently, with the vigorous development of the Internet of Things (IoT) technology, all kinds of intelligent home appliances in the market are constantly innovating. The public requirements for residential safety and convenience are also increasing. Meanwhile, with the improvement of indigenous medical technology and quality of life, people’s average lifespan is gradually increasing. However, countries around the world are facing the problem of aging societies. Hand gesture recognition is gaining popularity in the fields of gesture control, robotics, or medical applications. Therefore, how to create a convenient and smart control system of home appliances for the elderly or the disabled has become the objective of this study. It aims to use Google MediaPipe to develop a hand tracking system, which detected 21 key points of a hand through the camera lens of a mobile device and used a vector formula to calculate the angle of the intersection of two lines based on four key points. After the angle of bending finger is obtained, users’ hand gesture can be recognized. Our experiments have confirmed that the recognition precision and recall values of hand gesture for numbers 0–9 reached 98.80% and 97.67%, respectively; and the recognition results were used to control home appliances through the low-cost IoT-Enabled system.