A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection

Abstract

Background: Assembly guided by paper documents is the most widespread type used in the process of aircraft cable assembly. This process is very complicated and requires assembly workers with high-level skills. The technologies of wearable Augmented Reality (AR) and portable visual inspection can be exploited to improve the efficiency and the quality of cable assembly. Methods: In this study, we propose a smart assistance system for cable assembly that combines wearable AR with portable visual inspection. Specifically, a portable visual device based on binocular vision and deep learning is developed to realize fast detection and recognition of cable brackets that are installed on aircraft airframes. A Convolutional Neural Network (CNN) is then developed to read the texts on cables after images are acquired from the camera of the wearable AR device. An authoring tool that was developed to create and manage the assembly process is proposed to realize visual guidance of the cable assembly process based on a wearable AR device. The system is applied to cable assembly on an aircraft bulkhead prototype. Results: The results show that this system can recognize the number, types, and locations of brackets, and can correctly read the text of aircraft cables. The authoring tool can assist users who lack professional programming experience in establishing a process plan, i. e., assembly outline based on AR for cable assembly. Conclusions: The system can provide quick assembly guidance for aircraft cable with texts, images, and a 3D model. It is beneficial for reducing the dependency on paper documents, labor intensity, and the error rate. Keywords: Cable assembly, Visual inspection, Text reading, Wearable AR, Deep learning