Pre-trained convolutional neural network with transfer learning by artificial illustrated images classify power Doppler ultrasound images of rheumatoid arthritis joints

Abstract

Objective To study the classification performance of a pre-trained convolutional neural network (CNN) with transfer learning by artificial joint ultrasonography images in rheumatoid arthritis (RA). Methods This retrospective study focused on abnormal synovial vascularity and created 870 artificial joint ultrasound images based on the European League Against Rheumatism/Outcome Measure in Rheumatology scoring system. One CNN, the Visual Geometry Group (VGG)-16, was trained with transfer learning using the 870 artificial images for initial training and the original plus five additional images for second training. The models were then tested for the ability to classify joints using real joint ultrasound images obtained from patients with RA. The study was registered in UMIN Clinical Trials Registry (UMIN000054321). Results A total of 156 clinical joint ultrasound images from 74 patients with RA were included. The initial model showed moderate classification performance, but the area under curve (AUC) for grade 1 synovitis was particularly low (0.59). The second model showed improvement in classifying grade 1 synovitis (AUC 0.73). Conclusions Artificial images may be useful for training VGG-16. The present novel approach of using artificial images as an alternative to actual images for training a CNN has the potential to be applied in medical imaging fields that face difficulties in collecting real clinical images.