Deep Learning for the Automatic Diagnosis and Analysis of Bone Metastasis on Bone Scintigrams

Abstract

Simin Liu,1,* Ming Feng,2,* Tingting Qiao,1,* Haidong Cai,1 Kele Xu,3 Xiaqing Yu,1 Wen Jiang,1 Zhongwei Lv,1 Yin Wang,2 Dan Li1 1Department of Nuclear Medicine, Shanghai Tenth People’s Hospital, Tongji University School of Medicine, Shanghai, People’s Republic of China; 2School of Electronic and Information Engineering, Tongji University, Shanghai, People’s Republic of China; 3National Key Laboratory of Parallel and Distributed Processing, National University of Defense Technology, Changsha, People’s Republic of China*These authors contributed equally to this workCorrespondence: Zhongwei Lv; Dan LiDepartment of Nuclear Medicine, Shanghai Tenth People’s Hospital, Tongji University, Yanchangzhong Road 301, Shanghai, 200072, People’s Republic of ChinaTel [phone]Email [email protected]; [email protected]: To develop an approach for automatically analyzing bone metastases (BMs) on bone scintigrams based on deep learning technology.Methods: This research included a bone scan classification model, a regional segmentation model, an assessment model for tumor burden and a diagnostic report generation model. Two hundred eighty patients with BMs and 341 patients with non-BMs were involved. Eighty percent of cases were randomly extracted from two groups as training set. Remaining cases were as testing set. A deep residual convolutional neural network with different structures was used to determine whether metastatic bone lesions existed, regions of lesions were automatically segmented. Bone scan tumor burden index (BSTBI) was calculated; finally, diagnostic report could be automatically generated. The sensitivity, specificity and accuracy of classification model were compared with three physicians with different clinical experience. The Dice coefficient evaluated the effect of segmentation model and compared to the result of nnU-Net model. The correlation between BSTBI and blood alkaline phosphatase (ALP) level was analyzed to verify the efficiency of BSTBI. The performance of report generation model was evaluated by the accuracy of interpretation of report.Results: In testing set, the sensitivity, specificity and accuracy of classification model were 92.59%, 85.51% and 88.62%, respectively. The accuracy showed no statistical difference with moderately and experienced physicians and obviously outperformed the inexperienced. The Dice coefficient of BMs area was 0.7387 in segmentation stage. Based on the whole model frame, our segmentation model outperformed the nnU-Net. BSTBI value changed as the BMs changed. There was a positive correlation between BSTBI and ALP level. The accuracy of report generation model was 78.05%.Conclusion: Deep learning based on automatic analysis frameworks for BMs can accurately identify BMs, preliminarily realize a fully automatic analysis process from raw data to report generation. BSTBI can be used as a quantitative evaluation indicator to assess the effect of therapy on BMs in different patients or in the same patient before and after treatment.Keywords: bone metastases, bone scintigraphy, deep learning, tumor burden, automatic report generation

Keywords

• bone metastases · bone scintigraphy · deep learning ·tumor burden· automatic report generation