Rancang Bangun AirMouse Menggunakan Sarung Tangan Bersensor Berbasis ESP32

Abstract

Digital interactions are still commonly using indirect media such as mouse and keyboard to provide user input in the form of two-dimensional data. Therefore, to provide intuition in virtual interactions, it is possible to add media that can draw directly in the air or a flat surface that will track hand movements and overall finger position. In this research, we try to track hand movements in real time by capturing the position of the hand and finger curvature using a wearable sensor equipped with an Inertial Measurement Unit (IMU) sensor and a flex sensor installed by the user. Then the system will identify the position of the user's finger bending. and the location indicated by the sensors installed to move the cursor on the screen and simulate left-click and right-click hand movements as with a traditional mouse. By using this system, users can interact with the computer more naturally and get the accuracy of cursor movement with the accuracy of finger movement translation reaching more than 85% and the translation of hand movements to mouse cursor movements is on average 73% for shapes that use straight lines. and 23.4% on curved lines such as circles and other shapes.

Keywords

• hand movement recognition
• line drawing
• inertial measurement unit
• human-computer interaction