Comparison of AI software tools for automated detection, quantification and categorization of pulmonary nodules in the HANSE LCS trial

Abstract

Abstract Participant management in a lung cancer screening (LCS) depends on the assigned Lung Imaging Reporting and Data System (Lung-RADS) category, which is based on reliable detection and measurement of pulmonary nodules. The aim of this study was to compare the agreement of two AI-based software tools for detection, quantification and categorization of pulmonary nodules in an LCS program in Northern Germany (HANSE-trial). 946 low-dose baseline CT-examinations were analyzed by two AI software tools regarding lung nodule detection, quantification and categorization and compared to the final radiologist read. The relationship between detected nodule volumes by both software tools was assessed by Pearson correlation (r) and tested for significance using Wilcoxon signed-rank test. The consistency of Lung-RADS classifications between Software tool 1 (S1, Aview v2.5, Coreline Soft, Seoul, Korea) and Software tool 2 (S2, Prototype ‘’ChestCTExplore’’, software version ToDo, Siemens Healthineers, Forchheim, Germany) was evaluated by Cohen’s kappa (κ) and percentual agreement (PA).The derived volumes of true positive nodules were strongly correlated (r > 0.95), the volume derived by S2 was significantly higher than by S1 (P < 0.0001, mean difference: 6mm3). Moderate PA (62%) between S1 and S2 was found in the assignment of Lung-RADS classification (κ = 0.45). The PA of Lung-RADS classification to final read was 75% and 55% for S1 and S2, but the incorporation of S1 into the initial nodule detection and segmentation must be considered here. Significant nodule volume differences between AI software tools lead to different Lung-RADS scores in 38% of cases, which may result in altered participant management. Therefore, high performance and agreement of accredited AI software tools are necessary for a future national LCS program.