A Multi-Modal Machine Learning Methodology for Predicting Solitary Pulmonary Nodule Malignancy in Patients Undergoing PET/CT Examination

Abstract

This study explores a multi-modal machine-learning-based approach to classify solitary pulmonary nodules (SPNs). Non-small cell lung cancer (NSCLC), presenting primarily as SPNs, is the leading cause of cancer-related deaths worldwide. Early detection and appropriate management of SPNs are critical to improving patient outcomes, necessitating efficient diagnostic methodologies. While CT and PET scans are pivotal in the diagnostic process, their interpretation remains prone to human error and delays in treatment implementation. This study proposes a machine-learning-based network to mitigate these concerns, integrating CT, PET, and manually extracted features in a multi-modal manner by integrating multiple image modalities and tabular features). CT and PET images are classified by a VGG19 network, while additional SPN features in combination with the outputs of VGG19 are processed by an XGBoost model to perform the ultimate diagnosis. The proposed methodology is evaluated using patient data from the Department of Nuclear Medicine of the University Hospital of Patras in Greece. We used 402 patient cases with human annotations to internally validate the model and 96 histopathological-confirmed cases for external evaluation. The model exhibited 97% agreement with the human readers and 85% diagnostic performance in the external set. It also identified the VGG19 predictions from CT and PET images, SUVmax, and diameter as key malignancy predictors. The study suggests that combining all available image modalities and SPN characteristics improves the agreement of the model with the human readers and the diagnostic efficiency.

Keywords

• solitary pulmonary nodules
• computerized tomography
• positron emission tomography
• deep learning
• explainable artificial intelligence