Human-Robot Collaboration: An Augmented Reality Toolkit for Bi-Directional Interaction

Abstract

This work proposes an Augmented Reality (AR) application designed for HoloLens 2 which allows human operators, without particular experience or knowledge of robotics, to easily interact with collaborative robots. Building on the application presented in a previous work of the authors, the novel contributions are focused on a bi-directional interaction that manages the exchange of data from the robot to the human operator and, in the meantime, the flow of commands in the opposite direction. More in detail, the application includes the reading of the robot state, in terms of joint positions, velocities and torques, the visualization of the workspace and the generation and manipulation of the end-effector trajectory by directly moving a set of way-points displayed in the AR environment. Finally, the trajectory feasibility is verified and notified to the user by taking into account the workspace limits. A usability study of the AR platform has been conducted involving 45 participants with different ages and expertise in robot programming and Extended Reality (XR) platforms, comparing two programming methods: a classical kinesthetic teaching interface, provided by the Franka Emika Panda cobot, and the presented AR platform. Participants have reported the effectiveness of the proposed platform, experiencing less physical demand and higher intuitiveness and usability.

Keywords

• augmented reality
• robotics
• head-mounted display
• human-robot interaction