Spliced Multimodal Residual18 Neural Network: A Breakthrough in Pavement Crack Recognition and Categorization

Abstract

Machine learning and deep neural network developments have evolved image classification. This study offers a new neural network design with segmentation and localization, different crack types, categorization accuracy, and efficiency. In the proposed work, an advanced and innovative Spliced Multimodal Residual18 Neural Network (SMR18-NN) Model is presented that departs from previous models. For data authentication, augmentation is done by proposing the new and effective model known as an Augmented Minority Over-sampling Technique (AMOST). The SMR18 NN model combines the well-established ResNet18 framework with the Faster RCNN architecture, incorporating parts from the modified Fast RCNN and the Region Proposal Network. This work aims to revolutionize crack-type recognition and categorization in images. A Support Vector Machine strategically improves the network’s data classification. A modified ResNet18 model is ultimately implemented and compared with the proposed innovative SMR18-NN model. Both network’s parameters, such as epochs, number of iterations, etc., were kept the same for fair evaluation. Innovative frameworks and properly selected benchmark datasets supported this. The empirical results of this comparison study are convincing. ResNet18 training and testing accuracy was 90.60% and 85.20%, respectively. The SMR18-NN outperformed these results with 96.20 training and 92.00% testing accuracy. The experiment concluded with SMR18-NN accurately detecting image features, proving its superiority in image classification.

Keywords

• Faster RCNN
• ResNet18
• road crack image
• road cracks
• spliced multimodal Residual18 neural network
• localization