Teleoperation system for multiple robots with intuitive hand recognition interface

Abstract

Abstract Robotic teleoperation is essential for hazardous environments where human safety is at risk. However, efficient and intuitive human–machine interaction for multi-robot systems remains challenging. This article aims to demonstrate a robotic teleoperation system, denominated AutoNav, centered around autonomous navigation and gesture commands interpreted through computer vision. The central focus is on recognizing the palm of the hand as a control interface to facilitate human–machine interaction in the context of multi-robots. The MediaPipe framework was integrated to implement gesture recognition from a USB camera. The system was developed using the Robot Operating System, employing a simulated environment that includes the Gazebo and RViz applications with multiple TurtleBot 3 robots. The main results show a reduction of approximately 50% in the execution time, coupled with an increase in free time during teleoperation, reaching up to 94% of the total execution time. Furthermore, there is a decrease in collisions. These results demonstrate the effectiveness and practicality of the robotic control algorithm, showcasing its promise in managing teleoperations across multi-robots. This study fills a knowledge gap by developing a hand gesture-based control interface for more efficient and safer multi-robot teleoperation. These findings enhance human–machine interaction in complex robotic operations. A video showing the system working is available at https://youtu.be/94S4nJ3IwUw .

Keywords

• Autonomous navigation
• Human–robot
• Multi-robots
• ROS
• Teleoperation