Critical Care Explorations (Aug 2021)
Machine Learning Prediction of Death in Critically Ill Patients With Coronavirus Disease 2019
- Matthew M. Churpek, MD, MPH, PhD,
- Shruti Gupta, MD, MPH,
- Alexandra B. Spicer, MS,
- Salim S. Hayek, MD,
- Anand Srivastava, MD, MPH,
- Lili Chan, MD, MSCR,
- Michal L. Melamed, MD, MHS,
- Samantha K. Brenner, MD, MPH,
- Jared Radbel, MD,
- Farah Madhani-Lovely, MD,
- Pavan K. Bhatraju, MD, MSc,
- Anip Bansal, MD,
- Adam Green, MD, MBA,
- Nitender Goyal, MD,
- Shahzad Shaefi, MD, MPH,
- Chirag R. Parikh, MD, PhD,
- Matthew W. Semler, MD,
- David E. Leaf, MD, MMSc,
- Carl P. Walther,
- Samaya J. Anumudu,
- Justin Arunthamakun,
- Kathleen F. Kopecky,
- Gregory P. Milligan,
- Peter A. McCullough,
- ThuyDuyen Nguyen,
- Shahzad Shaefi,
- Megan L. Krajewski,
- Sidharth Shankar,
- Ameeka Pannu,
- Juan D. Valencia,
- Sushrut S. Waikar,
- Zoe A. Kibbelaar,
- Ambarish M. Athavale,
- Peter Hart,
- Oyintayo Ajiboye,
- Matthew Itteera,
- Adam Green,
- Jean-Sebastien Rachoin,
- Christa A. Schorr,
- Lisa Shea,
- Daniel L. Edmonston,
- Christopher L. Mosher,
- Alexandre M. Shehata,
- Zaza Cohen,
- Valerie Allusson,
- Gabriela Bambrick-Santoyo,
- Noor ul aain Bhatti,
- Bijal Metha,
- Aquino Williams,
- Samantha K. Brenner,
- Patricia Walters,
- Ronaldo C. Go,
- Keith M. Rose,
- Miguel A. Hernán,
- Amy M. Zhou,
- Ethan C. Kim,
- Rebecca Lisk,
- Lili Chan,
- Kusum S. Mathews,
- Steven G. Coca,
- Deena R. Altman,
- Aparna Saha,
- Howard Soh,
- Huei Hsun Wen,
- Sonali Bose,
- Emily Leven,
- Jing G. Wang,
- Gohar Mosoyan,
- Girish N. Nadkarni,
- Allon N. Friedman,
- John Guirguis,
- Rajat Kapoor,
- Christopher Meshberger,
- Chirag R. Parikh,
- Brian T. Garibaldi,
- Celia P. Corona-Villalobos,
- Yumeng Wen,
- Steven Menez,
- Rubab F. Malik,
- Carmen Elena Cervantes,
- Samir C. Gautam,
- Crystal Chang,
- H. Bryant Nguyen,
- Afshin Ahoubim,
- Leslie F. Thomas,
- Pramod K. Guru,
- Paul A. Bergl,
- Yan Zhou,
- Jesus Rodriguez,
- Jatan A. Shah,
- Mrigank S. Gupta,
- Princy N. Kumar,
- Deepa G. Lazarous,
- Seble G. Kassaye,
- Michal L. Melamed,
- Tanya S. Johns,
- Ryan Mocerino,
- Kalyan Prudhvi,
- Denzel Zhu,
- Rebecca V. Levy,
- Yorg Azzi,
- Molly Fisher,
- Milagros Yunes,
- Kaltrina Sedaliu,
- Ladan Golestaneh,
- Maureen Brogan,
- Jyotsana Thakkar,
- Neelja Kumar,
- Michael J. Ross,
- Michael Chang,
- Ritesh Raichoudhury,
- Edward J. Schenck,
- Soo Jung Cho,
- Maria Plataki,
- Sergio L. Alvarez-Mulett,
- Luis G. Gomez-Escobar,
- Di Pan,
- Stefi Lee,
- Jamuna Krishnan,
- William Whalen,
- David Charytan,
- Ashley Macina,
- Daniel W. Ross,
- Anand Srivastava,
- Alexander S. Leidner,
- Carlos Martinez,
- Jacqueline M. Kruser,
- Richard G. Wunderink,
- Alexander J. Hodakowski,
- Juan Carlos Q. Velez,
- Eboni G. Price-Haywood,
- Luis A. Matute-Trochez,
- Anna E. Hasty,
- Muner MB. Mohamed,
- Rupali S. Avasare,
- David Zonies,
- David E. Leaf,
- Shruti Gupta,
- Rebecca M. Baron,
- Meghan E. Sise,
- Erik T. Newman,
- Samah Abu Omar,
- Kapil K. Pokharel,
- Shreyak Sharma,
- Harkarandeep Singh,
- Simon Correa Gaviria,
- Tanveer Shaukat,
- Omer Kamal,
- Wei Wang,
- Heather Yang,
- Jeffery O. Boateng,
- Meghan Lee,
- Ian A. Strohbehn,
- Jiahua Li,
- Saif A. Muhsin,
- Ernest I. Mandel,
- Ariel L. Mueller,
- Nicholas S. Cairl,
- Farah Madhani-Lovely,
- Chris Rowan,
- Farah Madhai-Lovely,
- Vasil Peev,
- Jochen Reiser,
- John J. Byun,
- Andrew Vissing,
- Esha M. Kapania,
- Zoe Post,
- Nilam P. Patel,
- Joy-Marie Hermes,
- Anne K. Sutherland,
- Amee Patrawalla,
- Diana G. Finkel,
- Barbara A. Danek,
- Sowminya Arikapudi,
- Jeffrey M. Paer,
- Jared Radbel,
- Sonika Puri,
- Jag Sunderram,
- Matthew T. Scharf,
- Ayesha Ahmed,
- Ilya Berim,
- Jayanth Vatson,
- Shuchi Anand,
- Joseph E. Levitt,
- Pablo Garcia,
- Suzanne M. Boyle,
- Rui Song,
- Jingjing Zhang,
- Moh’d A. Sharshir,
- Vadym V. Rusnak,
- Anip Bansal,
- Amber S. Podoll,
- Michel Chonchol,
- Sunita Sharma,
- Ellen L. Burnham,
- Arash Rashidi,
- Rana Hejal,
- Eric Judd,
- Laura Latta,
- Ashita Tolwani,
- Timothy E. Albertson,
- Jason Y. Adams,
- Steven Y. Chang,
- Rebecca M. Beutler,
- Carl E. Schulze,
- Etienne Macedo,
- Harin Rhee,
- Kathleen D. Liu,
- Vasantha K. Jotwani,
- Jay L. Koyner,
- Chintan V. Shah,
- Vishal Jaikaransingh,
- Stephanie M. Toth-Manikowski,
- Min J. Joo,
- James P. Lash,
- Javier A. Neyra,
- Nourhan Chaaban,
- Alfredo Iardino,
- Elizabeth H. Au,
- Jill H. Sharma,
- Marie Anne Sosa,
- Sabrina Taldone,
- Gabriel Contreras,
- David De La Zerda,
- Hayley B. Gershengorn,
- Salim S. Hayek,
- Pennelope Blakely,
- Hanna Berlin,
- Tariq U. Azam,
- Husam Shadid,
- Michael Pan,
- Patrick O’ Hayer,
- Chelsea Meloche,
- Rafey Feroze,
- Kishan J. Padalia,
- Jeff Leya,
- John P. Donnelly,
- Andrew J. Admon,
- Jennifer E. Flythe,
- Matthew J. Tugman,
- Brent R. Brown,
- Amanda K. Leonberg-Yoo,
- Ryan C. Spiardi,
- Todd A. Miano,
- Meaghan S. Roche,
- Charles R. Vasquez,
- Amar D. Bansal,
- Natalie C. Ernecoff,
- Csaba P. Kovesdy,
- Miklos Z. Molnar,
- S. Susan Hedayati,
- Mridula V. Nadamuni,
- Sadaf S. Khan,
- Duwayne L. Willett,
- Samuel A.P. Short,
- Amanda D. Renaghan,
- Pavan Bhatraju,
- A. Bilal Malik,
- Matthew W. Semler,
- Anitha Vijayan,
- Christina Mariyam Joy,
- Tingting Li,
- Seth Goldberg,
- Patricia F. Kao,
- Greg L. Schumaker,
- Nitender Goyal,
- Anthony J. Faugno,
- Greg L. Schumaker,
- Caroline M. Hsu,
- Asma Tariq,
- Leah Meyer,
- Marta Christov,
- Francis P. Wilson,
- Tanima Arora,
- Ugochukwu Ugwuowo
Affiliations
- Matthew M. Churpek, MD, MPH, PhD
- 1 Division of Pulmonary and Critical Care, Department of Medicine, University of Wisconsin, Madison, WI.
- Shruti Gupta, MD, MPH
- 2 Division of Renal Medicine, Department of Medicine, Brigham and Women’s Hospital, Boston, MA.
- Alexandra B. Spicer, MS
- 1 Division of Pulmonary and Critical Care, Department of Medicine, University of Wisconsin, Madison, WI.
- Salim S. Hayek, MD
- 3 Division of Cardiology, Department of Medicine, University of Michigan, Ann Arbor, MI.
- Anand Srivastava, MD, MPH
- 4 Center for Translational Metabolism and Health, Institute for Public Health and Medicine, Department of Medicine, Division of Nephrology and Hypertension, Northwestern University Feinberg School of Medicine, Chicago, IL.
- Lili Chan, MD, MSCR
- 5 Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY.
- Michal L. Melamed, MD, MHS
- 6 Department of Medicine, Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY.
- Samantha K. Brenner, MD, MPH
- 7 Department of Internal Medicine, Hackensack Meridian School of Medicine, Seton Hall, NJ.
- Jared Radbel, MD
- 9 Department of Medicine, Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ.
- Farah Madhani-Lovely, MD
- 10 Department of Pulmonary and Critical Care Medicine, Renown Health, Reno, NV.
- Pavan K. Bhatraju, MD, MSc
- 11 Department of Medicine, Division of Pulmonary, Critical Care and Sleep Medicine, University of Washington, Seattle, WA.
- Anip Bansal, MD
- 12 Department of Medicine, Division of Renal Diseases and Hypertension, University of Colorado Anschutz Medical Campus Aurora, CO.
- Adam Green, MD, MBA
- 13 Department of Critical Care Medicine, Cooper University Health Care, Camden, NJ.
- Nitender Goyal, MD
- 14 Department of Medicine, Division of Nephrology, Tufts Medical Center, Boston, MA.
- Shahzad Shaefi, MD, MPH
- 15 Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Boston, MA.
- Chirag R. Parikh, MD, PhD
- 16 Department of Medicine, Division of Nephrology, Johns Hopkins School of Medicine, Baltimore, MD.
- Matthew W. Semler, MD
- 17 Department of Medicine, Division of Allergy, Pulmonary, and Critical Care Medicine, Vanderbilt University Medical Center, Nashville, TN.
- David E. Leaf, MD, MMSc
- 2 Division of Renal Medicine, Department of Medicine, Brigham and Women’s Hospital, Boston, MA.
- Carl P. Walther
- Samaya J. Anumudu
- Justin Arunthamakun
- Kathleen F. Kopecky
- Gregory P. Milligan
- Peter A. McCullough
- ThuyDuyen Nguyen
- Shahzad Shaefi
- Megan L. Krajewski
- Sidharth Shankar
- Ameeka Pannu
- Juan D. Valencia
- Sushrut S. Waikar
- Zoe A. Kibbelaar
- Ambarish M. Athavale
- Peter Hart
- Oyintayo Ajiboye
- Matthew Itteera
- Adam Green
- Jean-Sebastien Rachoin
- Christa A. Schorr
- Lisa Shea
- Daniel L. Edmonston
- Christopher L. Mosher
- Alexandre M. Shehata
- Zaza Cohen
- Valerie Allusson
- Gabriela Bambrick-Santoyo
- Noor ul aain Bhatti
- Bijal Metha
- Aquino Williams
- Samantha K. Brenner
- Patricia Walters
- Ronaldo C. Go
- Keith M. Rose
- Miguel A. Hernán
- Amy M. Zhou
- Ethan C. Kim
- Rebecca Lisk
- Lili Chan
- Kusum S. Mathews
- Steven G. Coca
- Deena R. Altman
- Aparna Saha
- Howard Soh
- Huei Hsun Wen
- Sonali Bose
- Emily Leven
- Jing G. Wang
- Gohar Mosoyan
- Girish N. Nadkarni
- Allon N. Friedman
- John Guirguis
- Rajat Kapoor
- Christopher Meshberger
- Chirag R. Parikh
- Brian T. Garibaldi
- Celia P. Corona-Villalobos
- Yumeng Wen
- Steven Menez
- Rubab F. Malik
- Carmen Elena Cervantes
- Samir C. Gautam
- Crystal Chang
- H. Bryant Nguyen
- Afshin Ahoubim
- Leslie F. Thomas
- Pramod K. Guru
- Paul A. Bergl
- Yan Zhou
- Jesus Rodriguez
- Jatan A. Shah
- Mrigank S. Gupta
- Princy N. Kumar
- Deepa G. Lazarous
- Seble G. Kassaye
- Michal L. Melamed
- Tanya S. Johns
- Ryan Mocerino
- Kalyan Prudhvi
- Denzel Zhu
- Rebecca V. Levy
- Yorg Azzi
- Molly Fisher
- Milagros Yunes
- Kaltrina Sedaliu
- Ladan Golestaneh
- Maureen Brogan
- Jyotsana Thakkar
- Neelja Kumar
- Michael J. Ross
- Michael Chang
- Ritesh Raichoudhury
- Edward J. Schenck
- Soo Jung Cho
- Maria Plataki
- Sergio L. Alvarez-Mulett
- Luis G. Gomez-Escobar
- Di Pan
- Stefi Lee
- Jamuna Krishnan
- William Whalen
- David Charytan
- Ashley Macina
- Daniel W. Ross
- Anand Srivastava
- Alexander S. Leidner
- Carlos Martinez
- Jacqueline M. Kruser
- Richard G. Wunderink
- Alexander J. Hodakowski
- Juan Carlos Q. Velez
- Eboni G. Price-Haywood
- Luis A. Matute-Trochez
- Anna E. Hasty
- Muner MB. Mohamed
- Rupali S. Avasare
- David Zonies
- David E. Leaf
- Shruti Gupta
- Rebecca M. Baron
- Meghan E. Sise
- Erik T. Newman
- Samah Abu Omar
- Kapil K. Pokharel
- Shreyak Sharma
- Harkarandeep Singh
- Simon Correa Gaviria
- Tanveer Shaukat
- Omer Kamal
- Wei Wang
- Heather Yang
- Jeffery O. Boateng
- Meghan Lee
- Ian A. Strohbehn
- Jiahua Li
- Saif A. Muhsin
- Ernest I. Mandel
- Ariel L. Mueller
- Nicholas S. Cairl
- Farah Madhani-Lovely
- Chris Rowan
- Farah Madhai-Lovely
- Vasil Peev
- Jochen Reiser
- John J. Byun
- Andrew Vissing
- Esha M. Kapania
- Zoe Post
- Nilam P. Patel
- Joy-Marie Hermes
- Anne K. Sutherland
- Amee Patrawalla
- Diana G. Finkel
- Barbara A. Danek
- Sowminya Arikapudi
- Jeffrey M. Paer
- Jared Radbel
- Sonika Puri
- Jag Sunderram
- Matthew T. Scharf
- Ayesha Ahmed
- Ilya Berim
- Jayanth Vatson
- Shuchi Anand
- Joseph E. Levitt
- Pablo Garcia
- Suzanne M. Boyle
- Rui Song
- Jingjing Zhang
- Moh’d A. Sharshir
- Vadym V. Rusnak
- Anip Bansal
- Amber S. Podoll
- Michel Chonchol
- Sunita Sharma
- Ellen L. Burnham
- Arash Rashidi
- Rana Hejal
- Eric Judd
- Laura Latta
- Ashita Tolwani
- Timothy E. Albertson
- Jason Y. Adams
- Steven Y. Chang
- Rebecca M. Beutler
- Carl E. Schulze
- Etienne Macedo
- Harin Rhee
- Kathleen D. Liu
- Vasantha K. Jotwani
- Jay L. Koyner
- Chintan V. Shah
- Vishal Jaikaransingh
- Stephanie M. Toth-Manikowski
- Min J. Joo
- James P. Lash
- Javier A. Neyra
- Nourhan Chaaban
- Alfredo Iardino
- Elizabeth H. Au
- Jill H. Sharma
- Marie Anne Sosa
- Sabrina Taldone
- Gabriel Contreras
- David De La Zerda
- Hayley B. Gershengorn
- Salim S. Hayek
- Pennelope Blakely
- Hanna Berlin
- Tariq U. Azam
- Husam Shadid
- Michael Pan
- Patrick O’ Hayer
- Chelsea Meloche
- Rafey Feroze
- Kishan J. Padalia
- Jeff Leya
- John P. Donnelly
- Andrew J. Admon
- Jennifer E. Flythe
- Matthew J. Tugman
- Brent R. Brown
- Amanda K. Leonberg-Yoo
- Ryan C. Spiardi
- Todd A. Miano
- Meaghan S. Roche
- Charles R. Vasquez
- Amar D. Bansal
- Natalie C. Ernecoff
- Csaba P. Kovesdy
- Miklos Z. Molnar
- S. Susan Hedayati
- Mridula V. Nadamuni
- Sadaf S. Khan
- Duwayne L. Willett
- Samuel A.P. Short
- Amanda D. Renaghan
- Pavan Bhatraju
- A. Bilal Malik
- Matthew W. Semler
- Anitha Vijayan
- Christina Mariyam Joy
- Tingting Li
- Seth Goldberg
- Patricia F. Kao
- Greg L. Schumaker
- Nitender Goyal
- Anthony J. Faugno
- Greg L. Schumaker
- Caroline M. Hsu
- Asma Tariq
- Leah Meyer
- Marta Christov
- Francis P. Wilson
- Tanima Arora
- Ugochukwu Ugwuowo
- DOI
- https://doi.org/10.1097/CCE.0000000000000515
- Journal volume & issue
-
Vol. 3,
no. 8
p. e0515
Abstract
OBJECTIVES:. Critically ill patients with coronavirus disease 2019 have variable mortality. Risk scores could improve care and be used for prognostic enrichment in trials. We aimed to compare machine learning algorithms and develop a simple tool for predicting 28-day mortality in ICU patients with coronavirus disease 2019. DESIGN:. This was an observational study of adult patients with coronavirus disease 2019. The primary outcome was 28-day inhospital mortality. Machine learning models and a simple tool were derived using variables from the first 48 hours of ICU admission and validated externally in independent sites and temporally with more recent admissions. Models were compared with a modified Sequential Organ Failure Assessment score, National Early Warning Score, and CURB-65 using the area under the receiver operating characteristic curve and calibration. SETTING:. Sixty-eight U.S. ICUs. PATIENTS:. Adults with coronavirus disease 2019 admitted to 68 ICUs in the United States between March 4, 2020, and June 29, 2020. INTERVENTIONS:. None. MEASUREMENTS AND MAIN RESULTS:. The study included 5,075 patients, 1,846 (36.4%) of whom died by day 28. eXtreme Gradient Boosting had the highest area under the receiver operating characteristic curve in external validation (0.81) and was well-calibrated, while k-nearest neighbors were the lowest performing machine learning algorithm (area under the receiver operating characteristic curve 0.69). Findings were similar with temporal validation. The simple tool, which was created using the most important features from the eXtreme Gradient Boosting model, had a significantly higher area under the receiver operating characteristic curve in external validation (0.78) than the Sequential Organ Failure Assessment score (0.69), National Early Warning Score (0.60), and CURB-65 (0.65; p < 0.05 for all comparisons). Age, number of ICU beds, creatinine, lactate, arterial pH, and Pao2/Fio2 ratio were the most important predictors in the eXtreme Gradient Boosting model. CONCLUSIONS:. eXtreme Gradient Boosting had the highest discrimination overall, and our simple tool had higher discrimination than a modified Sequential Organ Failure Assessment score, National Early Warning Score, and CURB-65 on external validation. These models could be used to improve triage decisions and clinical trial enrichment.