Validation of a Novel Mobile Application for Assessing Pediatric Tracheostomy Emergency Simulations
Marc‐Mina Tawfik,
Elliot Schiff,
Roxanna Mosavian,
Christine Campisi,
Amanda Shen,
Juan Lin,
Alanna M. Windsor,
Jacqueline Weingarten‐Arams,
Sara H. Soshnick,
Akira Nishisaki,
Sangmo Je,
Tensing Maa,
Ilana Harwayne‐Gidansky,
Regine M. Fortunov,
Christina J. Yang,
for the International Network for Simulation‐based Pediatric Innovation, Research, and Education (INSPIRE) Investigators
Affiliations
Marc‐Mina Tawfik
Albert Einstein College of Medicine Bronx New York USA
Elliot Schiff
Albert Einstein College of Medicine Bronx New York USA
Roxanna Mosavian
Albert Einstein College of Medicine Bronx New York USA
Christine Campisi
Albert Einstein College of Medicine Bronx New York USA
Amanda Shen
Albert Einstein College of Medicine Bronx New York USA
Juan Lin
Albert Einstein College of Medicine Bronx New York USA
Alanna M. Windsor
Albert Einstein College of Medicine Bronx New York USA
Jacqueline Weingarten‐Arams
Albert Einstein College of Medicine Bronx New York USA
Sara H. Soshnick
Albert Einstein College of Medicine Bronx New York USA
Akira Nishisaki
University of Pennsylvania Perelman School of Medicine Philadelphia Pennsylvania USA
Sangmo Je
Department of Anesthesiology and Critical Care Medicine, Division of Pediatric Critical Care Medicine Children's Hospital of Philadelphia Philadelphia Pennsylvania USA
Tensing Maa
Ohio State University College of Medicine Columbus Ohio USA
Ilana Harwayne‐Gidansky
Department of Pediatrics, Division of Pediatric Critical Care Medicine, The Bernard & Millie Duker Children's Hospital Albany Medical Center Albany New York USA
Regine M. Fortunov
Department of Pediatrics, Division of Neonatology Baylor College of Medicine Houston Texas USA
Christina J. Yang
Albert Einstein College of Medicine Bronx New York USA
for the International Network for Simulation‐based Pediatric Innovation, Research, and Education (INSPIRE) Investigators
Abstract Objective Pediatric tracheostomy is associated with high morbidity and mortality, yet clinician knowledge and quality of tracheostomy care may vary widely. In situ simulation is effective at detecting and mitigating related latent safety threats, but evaluation via retrospective video review has disadvantages (eg, delayed analysis, and potential data loss). We evaluated whether a novel mobile application is accurate and reliable for assessment of in situ tracheostomy emergency simulations. Methods A novel mobile application was developed for assessment of tracheostomy emergency in situ simulation team performance. After 1.25 hours of training, 6 raters scored 10 tracheostomy emergency simulation videos for the occurrence and timing of 12 critical steps. To assess accuracy, rater scores were compared to a reference standard to determine agreement for occurrence or absence of critical steps and a timestamp within ±5 seconds. Interrater reliability was determined through Cohen's and Fleiss' kappa and intraclass correlation coefficient. Results Raters had 86.0% agreement with the reference standard when considering step occurrence and timing, and 92.8% agreement when considering only occurrence. The average timestamp difference from the reference standard was 1.3 ± 18.5 seconds. Overall interrater reliability was almost perfect for both step occurrence (Fleiss' kappa of 0.81) and timing of step (intraclass correlation coefficient of 0.99). Discussion Using our novel mobile application, raters with minimal training accurately and reliably assessed videos of tracheostomy emergency simulations and identified areas for future refinement. Implications for Practice With refinements, this innovative mobile application is an effective tool for real‐time data capture of time‐critical steps in in situ tracheostomy emergency simulations.
