Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record

Chris J. Kennedy, PhD; Catherine Chiu, MD; Allyson Cook Chapman, MD; Oksana Gologorskaya, MS; Hassan Farhan, MD; Mary Han, MD; MacGregor Hodgson, MD; Daniel Lazzareschi, MD; Deepshikha Ashana, MD, MBA, MS; Sei Lee, MD; Alexander K. Smith, MD, MPH; Edie Espejo, MA; John Boscardin, PhD; Romain Pirracchio, MD, PhD; Julien Cobert, MD

doi:10.1097/CCE.0000000000000960

Critical Care Explorations (Oct 2023)

Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record

Chris J. Kennedy, PhD,
Catherine Chiu, MD,
Allyson Cook Chapman, MD,
Oksana Gologorskaya, MS,
Hassan Farhan, MD,
Mary Han, MD,
MacGregor Hodgson, MD,
Daniel Lazzareschi, MD,
Deepshikha Ashana, MD, MBA, MS,
Sei Lee, MD,
Alexander K. Smith, MD, MPH,
Edie Espejo, MA,
John Boscardin, PhD,
Romain Pirracchio, MD, PhD,
Julien Cobert, MD

Affiliations

Chris J. Kennedy, PhD: 1 Department of Psychiatry, Harvard Medical School, Boston, MA.
Catherine Chiu, MD: 3 Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, CA.
Allyson Cook Chapman, MD: 4 Critical Care and Palliative Medicine, Department of Internal Medicine, University of California San Francisco, San Francisco, CA.
Oksana Gologorskaya, MS: 6 Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, CA.
Hassan Farhan, MD: 7 Department of Anesthesiology, Perioperative and Pain Management, Stanford University, Stanford, CA.
Mary Han, MD: 2 Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA.
MacGregor Hodgson, MD: 2 Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA.
Daniel Lazzareschi, MD: 2 Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA.
Deepshikha Ashana, MD, MBA, MS: 8 Division of Pulmonary, Allergy, and Critical Care Medicine, Duke University, Durham, NC.
Sei Lee, MD: 9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.
Alexander K. Smith, MD, MPH: 9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.
Edie Espejo, MA: 9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.
John Boscardin, PhD: 9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.
Romain Pirracchio, MD, PhD: 3 Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, CA.
Julien Cobert, MD: 3 Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, CA.

DOI: https://doi.org/10.1097/CCE.0000000000000960
Journal volume & issue: Vol. 5, no. 10
p. e0960

Abstract

Read online

OBJECTIVES:. To develop proof-of-concept algorithms using alternative approaches to capture provider sentiment in ICU notes. DESIGN:. Retrospective observational cohort study. SETTING:. The Multiparameter Intelligent Monitoring of Intensive Care III (MIMIC-III) and the University of California, San Francisco (UCSF) deidentified notes databases. PATIENTS:. Adult (≥18 yr old) patients admitted to the ICU. MEASUREMENTS AND MAIN RESULTS:. We developed two sentiment models: 1) a keywords-based approach using a consensus-based clinical sentiment lexicon comprised of 72 positive and 103 negative phrases, including negations and 2) a Decoding-enhanced Bidirectional Encoder Representations from Transformers with disentangled attention-v3-based deep learning model (keywords-independent) trained on clinical sentiment labels. We applied the models to 198,944 notes across 52,997 ICU admissions in the MIMIC-III database. Analyses were replicated on an external sample of patients admitted to a UCSF ICU from 2018 to 2019. We also labeled sentiment in 1,493 note fragments and compared the predictive accuracy of our tools to three popular sentiment classifiers. Clinical sentiment terms were found in 99% of patient visits across 88% of notes. Our two sentiment tools were substantially more predictive (Spearman correlations of 0.62–0.84, p values < 0.00001) of labeled sentiment compared with general language algorithms (0.28–0.46). CONCLUSION:. Our exploratory healthcare-specific sentiment models can more accurately detect positivity and negativity in clinical notes compared with general sentiment tools not designed for clinical usage.

Published in Critical Care Explorations

ISSN: 2639-8028 (Online)
Publisher: Wolters Kluwer
Country of publisher: United States
LCC subjects: Medicine: Internal medicine: Medical emergencies. Critical care. Intensive care. First aid
Website: https://journals.lww.com/ccejournal/pages/default.aspx

About the journal