Assessment of Operator Reliability in Ultrasound of the Median and Ulnar Nerve Using Bland-Altman Analysis

Abstract

Currently, there is no standardized method to evaluate operator reliability in nerve ultrasound. A short prospective protocol using Bland–Altman analysis was developed to assess the level of agreement between operators with different expertise levels. A control rater without experience in nerve ultrasound, three novices after two months of training, an experienced rater with two years of experience, and a reference rater performed blinded ultrasound examinations of the left median and ulnar nerve in 42 nerve sites in healthy volunteers. The precision of Bland–Altman agreement analysis was tested using the Preiss–Fisher procedure. Intraclass correlation coefficients (ICC), coefficients of variation, and Bland–Altman limits of agreement were calculated. The sample size calculation and Preiss–Fisher procedure showed a sufficient precision of Bland–Altman agreement analysis. Limits of agreement of all trained novices ranged from 2.0 to 2.9 mm2 and were within the test’s maximum tolerated difference. Ninety-five percent confidence intervals of limits of agreement revealed a higher precision in the experienced rater’s measurements. Operator reliability in nerve ultrasound of the median and ulnar nerve arm nerves can be evaluated with a short prospective controlled protocol using Bland–Altman statistics, allowing a clear distinction between an untrained rater, trained novices after two months of training, and an experienced rater.

Keywords

• nerve ultrasound
• cross-sectional area
• reliability
• training effect
• Bland–Altman agreement analysis