Effect of CAD on performance in ASPECTS reading

Abstract

Background and purpose: While computer-aided diagnosis (CAD) tools are widely used in stroke imaging routines already, their influence on actual decision-making is still underexplored. We analyzed the effect of a simulated CAD tool on ASPECT-Scoring on acute-stroke CT-scans with respect to experience level. Materials and methods: Baseline CT scans of 100 stroke patients from the MR CLEAN trial with consensus-ASPECTS as ground truth were independently ASPECTS graded by three readers with different levels of experience. Weeks later the same CTs were re-analyzed with additional displaying of simulated ASPECTS (s-ASPECTS, by adding or subtracting 2 points from the ground truth). Readers were told that the score was generated by an automatic ASPECT-Scoring algorithm. The influence of the displayed s-ASPECTS on the readers’ second ASPECT-Scoring was analyzed by using a linear mixed model and the reliability was assessed. Performance was measured as the absolute difference between readers ASPECTS and consensus-ASPECTS. Results: The influence of the s-ASPECTS on the second ASPECT-Scoring was the lowest for the reader with the most experience in neuroradiology, while the other readers were significantly more influenced. All readers veered further away from the ground truth in their second ASPECT-Scoring with the s-ASPECTS, though not significantly. Overall interrater reliability was excellent (ICC = 0.94 [0.92–0.96]). Conclusions: ASPECT-Scoring may be significantly influenced by simulated ASPECTS displayed by a suboptimal CAD tool, especially in readers with less experience, and performance tends to decrease. Keywords: Computed tomography, Ischemic stroke, Diagnostic method, Alberta stroke program early CT score, Machine learning