Abstract Number ‐ 182: Practical performance of stroke detection by Rapid and Viz.AI artificial intelligence applications.

Abstract

Introduction Artificial intelligence applications for detecting ischemic strokes have recently been developed (Murray et al. 2020; Lotan 2021). Multiple applications exist that use proprietary algorithms to detect large vessel occlusions (LVOs) from CTA and CTP studies (Murray et al. 2020; Soun et al. 2021). These applications are intended to aid clinicians in diagnosing and triaging LVOs in anterior cranial arteries. Rapid (Ischemaview, Menlo Park, CA) and Viz.AI (Viz.AI, San Francisco, CA) are two such AI applications (Murray et al. 2020). The performance of both Rapid and Viz.AI in detecting LVOs in anterior circulation have been evaluated (Dehkharghani et al. 2021; Yahov‐Dovrat et al. 2021). These studies found Rapid and Viz.AI had high sensitivity (96% and 81% respectively) and high specificity (98% and 96% respectively) (Dehkharghani et al. 2021; Yahov‐Dovrat et al. 2021). In practice, CTA scans are sent to Rapid and Viz.AI for all patients suspected of having a stroke. We sought to evaluate the performance of Rapid and Viz.AI as they are used in practice, by measuring the performance of Rapid and Viz.AI LVO detection in cervical and intracranial arteries, in anterior cranial circulation, and in only extracranial and posterior vessels. Methods We used data from one large midwestern health system that uses both Rapid and Viz to conduct a retrospective cohort study to compare Rapid and Viz. We collected data from 672 patients that had CTA scans sent to both rapid and Viz between January 1st and December 31st. We manually abstracted from radiologists’ notes whether patients had an LVO in the vessels of the head and neck. We compared these data to records obtained from Rapid and Viz.AI of whether each application detected an LVO. We calculated the specificity and sensitivity of LVO detection for Rapid and Viz for all vessels evaluated in head and neck CTAs, for only the anterior intracranial vessels, and for only extracranial and posterior vessels. Results When including all vessels, Rapid had a sensitivity of 0.38 and specificity of 0.89, and Viz had sensitivity of 0.36 and specificity of 0.95. When including only anterior intracranial vessels, Rapid had a sensitivity of 0.62 and specificity of 0.89, and Viz had a sensitivity of 0.63 and specificity of 0.95. When including only extracranial and posterior vessels, Rapid had a sensitivity of 0.13 and sensitivity of 0.89, and Viz.AI had a sensitivity of 0.10 and specificity of 0.95. Conclusions We found specificity and sensitivity for Rapid and Viz.AI were lower than published, even when we restricted our analyses to the vessels in anterior circulation indicated for their use. Our study was not as rigorously controlled as other studies that have evaluated the performance of Rapid and Viz.AI, but we intended our results to evaluate the performance of these applications under actual use by clinicians. Our results suggest Rapid and Viz.AI could both aid in detecting anterior circulation clots, but radiologist interpretation of CTAs must still be relied upon to diagnose strokes, even in the anterior vessels for which their use is indicated.