Evaluating the performance of artificial intelligence-based software for digital mammography characterization

Abstract

BACKGROUND: Digital screening mammography is a key modality for early detection of breast cancer, reducing mortality by 20–40%. Many artificial intelligence (AI)-based services have been developed to automate the analysis of imaging data. AIM: The aim of the study was to compare mammography assessments using three types of AI services in multiple versions with radiologists’ conclusions. MATERIALS AND METHODS: Binary mammography scoring scales were compared with several types and versions of AI services regarding diagnostic accuracy, Matthews correlation coefficient, and maximum Youden’s index. RESULTS: A comparative analysis showed that the use of a binary scale for evaluating digital mammography affects the number of detected abnormalities and accuracy of AI results. In addition, diagnostic accuracy was found to be threshold dependent. AI Service 1 in version 3 had the best performance, as confirmed by most diagnostic accuracy parameters. CONCLUSIONS: Our results can be used to select AI services for interpreting mammography screening data. Using Youden’s index maximization to set up an AI service provides a balance of sensitivity and specificity that is not always clinically relevant.

Keywords

• malignant tumors of breast
• digital mammography
• artificial intelligence services
• diagnostic accuracy
• youden’s index