Can clinical vignettes administered by mobile phone call be used to measure quality of contraceptive care practices in Malawi?

Abstract

# Background Accurate and timely measures of quality of contraceptive care are required for program improvement but can be expensive and time-consuming to implement in low- and middle-income countries (LMICs). Clinical vignettes (CLVs) measure quality knowledge and have been shown to be a valid proxy of provider quality practice in the United States, and may be administered via mobile-phone, making them an attractive cost-savings alternative to field assessments. This cross-sectional study aims to validate mobile-phone CLVs for measuring quality of contraceptive service provision using simulated, “mystery” clients as the gold standard. # Methods Researchers trained to simulate clients (SC) seeking contraceptive care visited 112 government-administered clinics in six districts of Malawi. The providers who saw the simulated clients were called via mobile phone approximately three weeks later for CLV administration. We calculated sensitivity, specificity, and area under the ROC receiver operating characteristic curve (AUC) and compared the proportion of quality behaviors measured from both methods. # Results Overall, the quality measured by CLVs differed from quality measured with SCs. A higher proportion of providers reported clinical assessments and counseling on more topics during the CLV compared to the SCs, including counseling on what to do if the pill was not taken on time (59% vs. 26%; *P*<0.01). During the SC, providers asked more questions when taking client history and recommended the preferred method of the case scenario more often (81% vs. 49%; *P*<0.01). # Conclusions The quality of care ascertained through mobile-phone CLV was not similar enough to quality ascertained through SCs to replace them. Mobile phone-based CLV also failed to capture all areas of quality performance. Continued efforts are needed to identify and test innovative methods to accurately and inexpensively measure quality of care to inform improvement of provider performance.