Diabetes, Metabolic Syndrome and Obesity (Jul 2021)
External Validation of the Prognostic Prediction Model for 4-Year Risk of Metabolic Syndrome in Adults: A Retrospective Cohort Study
Abstract
Hui Zhang,1 Dandan Chen,1 Jin Shao,1 Ping Zou,2 Nianqi Cui,3 Leiwen Tang,1 Xiyi Wang,4 Dan Wang,1 Zhihong Ye1 1Department of Nursing, Zhejiang University School of Medicine Sir Run Run Shaw Hospital, Hangzhou, Zhejiang, People’s Republic of China; 2School of Nursing, Nipissing University, Toronto, Ontario, Canada; 3Department of Nursing, The Second Affiliated Hospital Zhejiang University School of Medicine, Hangzhou, Zhejiang, People’s Republic of China; 4School of Nursing, Shanghai JiaoTong University, Shanghai, People’s Republic of ChinaCorrespondence: Zhihong YeDepartment of Nursing, Zhejiang University School of Medicine Sir Run Run Shaw Hospital, Qingchun Dong Road, Jianggan District, Hangzhou, Zhejiang, People’s Republic of ChinaTel +86 13606612119Email [email protected]: A prediction model for 4-year risk of metabolic syndrome in adults was previously developed and internally validated. However, external validity or generalizability for this model was not assessed so it is not appropriate for clinical application. We aimed to externally validate this model based on a retrospective cohort.Patients and Methods: A retrospective cohort design and a temporal validation strategy were used in this study based on a dataset from 1 January 2015 to 31 December 2018. Multiple imputation was used for missing values. Model performance was evaluated by using discrimination, calibration (calibration plot, calibration slope, and calibration intercept), overall performance (Brier score), and decision curve analysis.Results: In external validation, the C-statistic was 0.782 (95% CI, 0.771– 0.793). The calibration plot shows good calibration, calibration slope was 1.006 (95% CI, − 0.011– 1.063), and calibration intercept was − 0.045 (95% CI, − 0.113– 0.022). Brier score was 0.164.The discrimination and calibration of the prediction model were good in temporal external validation.Conclusion: The discrimination and calibration of the prediction model were satisfactory in the temporal external validation. However, clinicians should be aware that this prediction model was developed and validated in a tertiary setting. It is strongly recommended that further studies validate this model in international cohorts and large, prospective cohorts in different institutions.Keywords: prediction model, prognosis, metabolic syndrome, algorithms, calibration, discrimination
