OSATS Assessment Instrument for Neonatal Lumbar Puncture

Abstract

Abstract Introduction Being procedurally competent to perform a lumbar puncture (LP) is an ACGME requirement for both emergency medicine and pediatric residents. There are few assessment tools with validity evidence for lumbar puncture. An objective structured assessment of technical skills for neonatal lumbar puncture (OSATS-LP) was developed and validity evidence was obtained in four of five areas of validity: content, response process, relation to other variables, and interrater reliability. Methods The development of this tool was part of a prospective study comparing pediatric resident performance of neonatal lumbar puncture with direct observation by faculty. T tests assessed significance between level of training, and previous ICU experience in comparison to their total performance score. The interrater agreement of the OSATS-LP was obtained using the Fleiss' Kappa for each domain. Results Sixteen pediatric residents completed the simulation with six raters evaluating each resident (96 ratings). The domains of sterility and cerebral spinal fluid collection had moderate statistical reliability (k = 0.41 and 0.51, respectively). The domains of preparation, analgesia, and management of labs had substantial reliability (k = 0.60, 0.62, and 0.62, respectively). The domains of positioning and needle insertion were less reliable (k = 0.16 and 0.16, respectively). Residents who had performed more LPs correlated with higher total score (Pearson's Correlation coefficient = 0.5, p < .05). Individuals who had completed one or more rotations in the neonatal intensive care unit had a higher total score (12.5 vs. 16.9; p < .01). The OSATS-LP has reasonable evidence in four of the five sources for test validity. Discussion This study serves as a launching point for using this tool in clinical environments such as the emergency department, and therefore, has the potential to provide real time formative and summative feedback to improve resident skills and ultimately lead to improvements in patient care. There are limited number of assessment tools for LP with validity evidence. This tool has demonstrated validity evidence in content, response process, and interrater reliability. This tool was successfully piloted in fifteen pediatric residents and showed increased competence in five of the seven domains with increasing PGY level.

Keywords

• Infant
• Neonate
• Newborn
• Lumbar Puncture
• Spinal Puncture
• Assessment