Masked autoencoder: influence of self-supervised pretraining on object segmentation in industrial images

Abstract

Abstract The amount of labelled data in industrial use cases is limited because the annotation process is time-consuming and costly. As in research, self-supervised pretraining such as MAE resulted in training segmentation models with fewer labels, this is also an interesting direction for industry. The reduction of required labels is achieved with large amounts of unlabelled images for the pretraining that aims to learn image features. This paper analyses the influence of MAE pretraining on the efficiency of label usage for semantic segmentation with UNETR. This is investigated for the use case of log-yard cranes. Additionally, two transfer learning cases with respect to crane type and perspective are considered in the context of label-efficiency. The results show that MAE is successfully applicable to the use case. With respect to the segmentation, an IoU improvement of 3.26% is reached while using 2000 labels. The strongest positive influence is found for all experiments in the lower label amounts. The highest effect is achieved with transfer learning regarding cranes, where IoU and Recall increase about 4.31% and 8.58%, respectively. Further analyses show that improvements result from a better distinction between the background and the segmented crane objects.

Keywords

• Masked autoencoder
• Self-supervised pretraining
• Semantic segmentation
• UNETR
• Label-efficiency
• Log-yard cranes