External validation and updating of published models for predicting 7-day risk of symptomatic intracranial hemorrhage after receiving alteplase for acute ischemic stroke: A retrospective cohort study

Abstract

Background: Prediction scores for symptomatic intracranial hemorrhage (sICH) in acute ischemic stroke patients receiving thrombolytic therapy have been widely developed, but the external validation of these scores, especially in the Thai population, is lacking. This study aims to externally validate existing models and update the selected model to enhance its performance in our specific context. Methods: This cohort study retrospectively collected data from medical records between 2013 and 2022. Acute ischemic stroke patients who received thrombolysis were included. All predictors were gathered at admission. External validation was performed on eight published prediction models; in addition, the observed and expected probabilities of sICH were compared. The most effective model for discrimination was then chosen for further updating using multivariable logistic regression and was bootstrapped for internal validation. Finally, a points-based system for clinical practice was developed from the optimism-corrected model. Results: Fifty patients (10% of the 502 included cohort members) experienced sICH after undergoing thrombolysis. The SICH score outperformed the other seven models in terms of discrimination (area under the receiver operating characteristic [AuROC] curve = 0.74 [95% confidence interval {CI} 0.67 to 0.81]), but it still overstated risk (expected-to-observed outcomes [E/O] ratio = 1.7). Once updated, the optimism-corrected revised SICH model showed somewhat better calibration (E/O = 1 and calibration-in-the-large = 0), slightly worse underprediction in the moderate-to-high risk group (calibration slope = 1.152), and marginally better discrimination (AuROC = 0.78). The points-based system also demonstrated substantial agreement (88.1%) with the risk groups predicted by the logistic regression model (kappa statistic = 0.78). Conclusion: Since the SICH score outperformed seven models in terms of discrimination, it was then modified to the Revised-SICH score, which predicted that patients with at least 5.5 points were at high risk of having sICH.

Keywords

• acute ischemic stroke
• external validation
• intravenous thrombolysis
• predictive risk score
• symptomatic intracranial hemorrhage