A Manual Assembly Virtual Training System With Automatically Generated Augmented Feedback: Using the Comparison of Digitized Operator’s Skill

Abstract

In general, an experienced operator or expert must deliver face-to-face industrial manual assembly (I-MA) training. There is a limited number and availability of expert trainers, making training less accessible to trainees. Industrial virtual training systems (I-VTS) from research communities and commercial systems are focused on providing task demonstrations with rich multimedia immersive content. These systems can significantly lessen the reliance on the expert, but they still require an expert to observe and provide feedback. This study presents the “EXpert Independent Manual AsseMbly Virtual Trainer” (EXAMINER), addressing these limitations by integrating vision-based digitization of I-MA skills, comparison of digitized operators, and providing augmented feedback assessing the training outcomes automatically. This approach enhances trainee access to training without expert dependency. The study discusses the rationale and implementation process for the EXAMINER framework and a simulated case study evaluating the EXAMINER framework’s ability to digitize and provide appropriate extrinsic augmented terminal feedback based on participant performance, marking a significant advancement in industrial training technology.

Keywords

• Industrial training
• deep learning
• digital twins
• pose estimation
• computer vision
• manual assembly