Annals of Intensive Care (May 2017)

External validation of SAPS 3 and MPM0-III scores in 48,816 patients from 72 Brazilian ICUs

  • Giulliana Martines Moralez,
  • Ligia Sarmet Cunha Farah Rabello,
  • Thiago Costa Lisboa,
  • Mariza da Fonte Andrade Lima,
  • Rodrigo Marques Hatum,
  • Fernando Vinicius Cesar De Marco,
  • Alessandra Alves,
  • Jorge Eduardo da Silva Soares Pinto,
  • Hélia Beatriz Nunes de Araújo,
  • Grazielle Viana Ramos,
  • Aline Reis Silva,
  • Guilherme Côrtes Fernandes,
  • Guilherme Brenande Alves Faria,
  • Ciro Leite Mendes,
  • Roberto Álvaro Ramos Filho,
  • Valdênia Pereira de Souza,
  • Pedro Emmanuel Alvarenga Americano do Brasil,
  • Fernando Augusto Bozza,
  • Jorge Ibrain Figueira Salluh,
  • Marcio Soares,
  • On Behalf of the ORCHESTRA Study Investigators

DOI
https://doi.org/10.1186/s13613-017-0276-3
Journal volume & issue
Vol. 7, no. 1
pp. 1 – 8

Abstract

Read online

Abstract Background The performance of severity-of-illness scores varies in different scenarios and must be validated prior of being used in a specific settings and geographic regions. Moreover, models’ calibration may deteriorate overtime and performance of such instruments should be reassessed regularly. Therefore, we aimed at to validate the SAPS 3 in a large contemporary cohort of patients admitted to Brazilian ICUs. In addition, we also compared the performance of the SAPS 3 with the MPM0-III. Methods This is a retrospective cohort study in which 48,816 (medical admissions = 67.9%) adult patients are admitted to 72 Brazilian ICUs during 2013. We evaluated models’ discrimination using the area under the receiver operating characteristic curve (AUROC). We applied the calibration belt to evaluate the agreement between observed and expected mortality rates (calibration). Results Mean SAPS 3 score was 44.3 ± 15.4 points. ICU and hospital mortality rates were 11.0 and 16.5%. We estimated predicted mortality using both standard (SE) and Central and South American (CSA) customized equations. Predicted mortality rates were 16.4 ± 19.3% (SAPS 3-SE), 21.7 ± 23.2% (SAPS 3-CSA) and 14.3 ± 14.0% (MPM0-III). Standardized mortality ratios (SMR) obtained for each model were: 1.00 (95% CI, 0.98–0.102) for the SAPS 3-SE, 0.75 (0.74–0.77) for the SAPS 3-CSA and 1.15 (1.13–1.18) for the MPM0-III. Discrimination was better for SAPS 3 models (AUROC = 0.85) than for MPM0-III (AUROC = 0.80) (p < 0.001). We applied the calibration belt to evaluate the agreement between observed and expected mortality rates (calibration): the SAPS 3-CSA overestimated mortality throughout all risk classes while the MPM0-III underestimated it uniformly. The SAPS 3-SE did not show relevant deviations from ideal calibration. Conclusions In a large contemporary database, the SAPS 3-SE was accurate in predicting outcomes, supporting its use for performance evaluation and benchmarking in Brazilian ICUs.

Keywords