Ultrasound identification of hepatic echinococcosis using a deep convolutional neural network model in China: a retrospective, large-scale, multicentre, diagnostic accuracy study

Abstract

Summary: Background: Ultrasonography is the most widely used technique to diagnose echinococcosis; however, challenges in using this technique and the demand on medical resources, especially in low-income or remote areas, can delay diagnosis. We aimed to develop a deep convolutional neural network (DCNN) model based on ultrasonography to identify echinococcosis and its types, especially alveolar echinococcosis. Methods: This retrospective, large-scale, multicentre study used ultrasound images from patients assessed at 84 hospitals in China, obtained between Jan 1, 2002, and Dec 31, 2021. Patients with a diagnosis of cystic echinococcosis, alveolar echinococcosis, or seven other types of focal liver lesions were included. We tested ResNet-50, ResNext-50, and VGG-16 as the backbone network architecture for a classification DCNN model and input the perinodular information from the ultrasound images. We trained and validated the DCNN model to diagnose and classify echinococcosis using still greyscale ultrasound images of focal liver lesions in four stages: differentiating between echinococcosis and other focal liver lesions (stage one); differentiating cystic echinococcosis, alveolar echinococcosis, and other focal liver lesions (stage two); differentiating cystic echinococcosis, alveolar echinococcosis, benign other focal liver lesions, and malignant focal liver lesions (stage three); and differentiating between active and transitional cystic echinococcosis and inactive cystic echinococcosis (stage four). We then tested the algorithm on internal, external, and prospective test datasets. The performance of DCNN was also compared with that of 12 radiologists recruited between Jan 15, 2022, and Jan 28, 2022, from Qinghai, Xinjiang, Anhui, Henan, Xizang, and Beijing, China, with different levels of diagnostic experience for echinococcosis and other focal liver lesions in a subset of ultrasound data that were randomly chosen from the prospective test dataset. The study is registered at ClinicalTrials.gov (NCT03871140). Findings: The study took place between Jan 1, 2002, and Dec 31, 2021. In total, to train and test the DCNN model, we used 9631 liver ultrasound images from 6784 patients (2819 [41·7%] female patients and 3943 [58·3%] male patients) from 87 Chinese hospitals. The DCNN model was trained with 6328 images, internally validated with 984 images, and tested with 2319 images. The ResNet-50 network architecture outperformed VGG-16 and ResNext-50 and was generalisable, with areas under the receiver operating characteristic curve (AUCs) of 0·982 (95% CI 0·960–0·994), 0·984 (0·972–0·992), and 0·913 (0·886–0·935) in distinguishing echinococcosis from other focal liver lesions; 0·986 (0·966–0·996), 0·962 (0·946–0·975), and 0·900 (0·872–0·924) in distinguishing alveolar echinococcosis from cystic echinococcosis and other focal liver lesions; and 0·974 (0·818–1·000), 0·956 (0·875–0·991), and 0·944 (0·844–0·988) in distinguishing active and transitional cystic echinococcosis from inactive echinococcosis in the three test datasets. Specifically, in patients with the hepatitis B or hepatitis C virus, the model could distinguish alveolar echinococcosis from hepatocellular carcinoma with an AUC of 0·892 (0·812–0·946). In identifying echinococcosis, the model showed significantly better performance compared with senior radiologists from a high-endemicity area (AUC 0·942 [0·904–0·967] vs 0·844 [0·820–0·866]; p=0·027) and improved the diagnostic ability of junior, attending, and senior radiologists before and after assistance with AI with comparison of AUCs of 0·743 (0·714–0·770) versus 0·850 (0·826–0·871); p<0·0001, 0·808 (0·782–0·832) versus 0·886 (0·864–0·905); p<0·0001, and 0·844 (0·820–0·866) versus 0·870 (0·847–0·890); p=0·092, respectively. Interpretation: The DCNN model was shown to be accurate and robust, and could improve the ultrasound diagnostic ability of radiologists for echinococcosis and its types for highly endemic and remote regions. Funding: National Natural Science Foundation of China and National Key Research & Development Program of China. Translation: For the Chinese translation of the abstract see Supplementary Materials section.