A New Model for Assembly Task Recognition: A Case Study of Seru Production System

Abstract

The Seru production system is an innovative assembly system that combines the flexibility of shop floor production and the high efficiency of assembly lines. In addition to its advantages, seru-type production has the disadvantage that all the tasks required for the assembly of a product are completed in a yatai by a cross-trained worker. This results in a higher risk of production errors compared to assembly lines. In order to mitigate these risks, a real-time control system is considered necessary. Recognition of assembly tasks is needed to prevent quality defects in the Seru production system. To meet this need, in this study, a skeleton-based deep learning hybrid Convolutional Neural Network-Bi-directional Gated Recurrent Unit-Convolutional Neural Network (CNN-BiGRU-CNN) assembly task recognition model is developed. In this study, a two-stage data augmentation approach is proposed to improve the performance of the model developed by utilizing the worker skeleton data obtained with Mediapipe Holistic infrastructure from the assembly video collected from the seru production system. The effectiveness of the model was evaluated by comparing it with Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Bi-directional Long Short Term Memory (BiLSTM), BiGRU, and hybrid models created by combinations of these models. The proposed data augmentation approach improved the performance of all the models compared in the study. With the proposed hybrid CNN-BiGRU-CNN model, the best performance was achieved with a prediction accuracy of 96%.

Keywords

• Assembly action recognition
• data augmentation
• deep learning
• MediaPipe
• pose estimation
• seru production system