Health Technology Assessment (Nov 2018)
Prognostic models for identifying risk of poor outcome in people with acute ankle sprains: the SPRAINED development and external validation study
Abstract
Background: Ankle sprains are very common injuries. Although recovery can occur within weeks, around one-third of patients have longer-term problems. Objectives: To develop and externally validate a prognostic model for identifying people at increased risk of poor outcome after an acute ankle sprain. Design: Development of a prognostic model in a clinical trial cohort data set and external validation in a prospective cohort study. Setting: Emergency departments (EDs) in the UK. Participants: Adults with an acute ankle sprain (within 7 days of injury). Sample size: There were 584 clinical trial participants in the development data set and 682 recruited for the external validation study. Predictors: Candidate predictor variables were chosen based on availability in the clinical data set, clinical consensus, face validity, a systematic review of the literature, data quality and plausibility of predictiveness of the outcomes. Main outcome measures: Models were developed to predict two composite outcomes representing poor outcome. Outcome 1 was the presence of at least one of the following symptoms at 9 months after injury: persistent pain, functional difficulty or lack of confidence. Outcome 2 included the same symptoms as outcome 1, with the addition of recurrence of injury. Rates of poor outcome in the external data set were lower than in the development data set, 7% versus 20% for outcome 1 and 16% versus 24% for outcome 2. Analysis: Multiple imputation was used to handle missing data. Logistic regression models, together with multivariable fractional polynomials, were used to select variables and identify transformations of continuous predictors that best predicted the outcome based on a nominal alpha of 0.157, chosen to minimise overfitting. Predictive accuracy was evaluated by assessing model discrimination (c-statistic) and calibration (flexible calibration plot). Results: (1) Performance of the prognostic models in development data set – the combined c-statistic for the outcome 1 model across the 50 imputed data sets was 0.74 [95% confidence interval (CI) 0.70 to 0.79], with good model calibration across the imputed data sets. The combined c-statistic for the outcome 2 model across the 50 imputed data sets was 0.70 (95% CI 0.65 to 0.74), with good model calibration across the imputed data sets. Updating these models, which used baseline data collected at the ED, with an additional variable at 4 weeks post injury (pain when bearing weight on the ankle) improved the discriminatory ability (c-statistic 0.77, 95% CI 0.73 to 0.82, for outcome 1 and 0.75, 95% CI 0.71 to 0.80, for outcome 2) and calibration of both models. (2) Performance of the models in the external data set – the combined c-statistic for the outcome 1 model across the 50 imputed data sets was 0.73 (95% CI 0.66 to 0.79), with a calibration plot intercept of –0.91 (95% CI –0.98 to 0.44) and slope of 1.13 (95% CI 0.76 to 1.50). The combined c-statistic for the outcome 2 model across the 50 imputed data sets was 0.63 (95% CI 0.58 to 0.69), with a calibration plot intercept of –0.25 (95% CI –0.27 to 0.11) and slope of 1.03 (95% CI 0.65 to 1.42). The updated models with the additional pain variable at 4 weeks had improved discriminatory ability over the baseline models but not better calibration. Conclusions: The SPRAINED (Synthesising a clinical Prognostic Rule for Ankle Injuries in the Emergency Department) prognostic models performed reasonably well, and showed benefit compared with not using any model; therefore, the models may assist clinical decision-making when managing and advising ankle sprain patients in the ED setting. The models use predictors that are simple to obtain. Limitations: The data used were from a randomised controlled trial and so were not originally intended to fulfil the aim of developing prognostic models. However, the data set was the best available, including data on the symptoms and clinical events of interest. Future work: Further model refinement, including recalibration or identifying additional predictors, may be required. The effect of implementing and using either model in clinical practice, in terms of acceptability and uptake by clinicians and on patient outcomes, should be investigated. Trial registration: Current Controlled Trials ISRCTN12726986. Funding: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 64. See the NIHR Journals Library website for further project information. Funding was also recieved from the NIHR Collaboration for Leadership in Applied Health Research, Care Oxford at Oxford Health NHS Foundation Trust, NIHR Biomedical Research Centre, Oxford, and the NIHR Fellowship programme.
Keywords
