Automatic classification of ultrasound thyroids images using vision transformers and generative adversarial networks

Abstract

Ultrasound has become the main imaging modality for the diagnosis of thyroid nodules. However, opinions issued and decisions taken by doctors are always subjective and depend on the experience of the doctor and his entourage. It is often essential to use a tool to guide doctors during decision-making. In this paper, we develop a new model, based on a deep learning approach, more specifically convolutional neural networks and vision transformers. Our goal was the classification of thyroid ultrasound images which were categorized into two classes (class: Benign and class: Malignant). At the beginning of this work, we implemented a Deep Convolutional Generative Adversarial Network (DCGAN) technique to fix the data shortage and the problem of the imbalanced data set. Then, we used two different methods for the splitting of the data, the first is the classic data split and the second is the cross-validation method. For the modeling techniques, we used two types of models such as: Convolutional Neural Network: (VGG16, EffecientNetB0 and ResNet50) and Vision Transformers: (ViT_B16 and Hybrid ViT). All the models were trained with Softmax and Support Vector Machine (SVM) classifiers. After evaluation of both the CNN and ViT models, we concluded that the SVM classification produces better performance than the Softmax classification for all of the models with the Hybrid ViT being on top in the matter of the metrics evaluation and the curves representation, the model achieved 97.63% Accuracy, resulting in a slight improvement over the existing studies. Our study achieved promising results, hopefully, this will help doctors better diagnose thyroid patients.

Keywords

• Thyroid nodules
• Vision transformer
• Transfer learning
• DCGAN
• Convolutional neural network
• Support vector machine (SVM)