Development of Standalone Extended-Reality-Supported Interactive Industrial Robot Programming System

Abstract

Extended reality (XR) is one of the most important technologies in developing a new generation of human–machine interfaces (HMIs). In this study, the design and implementation of a standalone interactive XR-supported industrial robot programming system using the Unity game engine is presented. The presented research aims to achieve a cross-platform solution that enables novel tools for robot programming, trajectory validation, and robot programming debugging within an extended reality environment. From a robotics perspective, key design tasks include modeling in the Unity environment based on robot CAD models and control design, which include inverse kinematics solution, trajectory planner development, and motion controller set-up. Furthermore, the integration of real-time vision, touchscreen interaction, and AR/VR headset interaction are involved within the overall system development. A comprehensive approach to integrating Unity with established industrial robot modeling conventions and control strategies is presented. The proposed modeling, control, and programming concepts, procedures, and algorithms are verified using a 6DoF robot with revolute joints. The benefits and challenges of using a standalone XR-supported interactive industrial robot programming system compared to integrated Unity–robotics development frameworks are discussed.

Keywords

• robot programming
• XR
• augmented reality
• virtual reality
• Unity
• physics engine