Survey implementation process and interviewer effects on skipping sequence of maternal and child health indicators from National Family Health Survey: An application of cross-classified multilevel model

Abstract

Implementing a large-scale survey involves a string of intricate procedures exposed to numerous types of survey errors. Uniform and systematic training protocols, comprehensive survey manuals, and multilayer supervision during survey implementation help reduce survey errors, providing a consistent fieldwork environment that should not result in any variation in the quality of data collected across interviewers and teams. With this background, the present study attempts to delineate the effect of field investigator (FI) teams and survey implementation design on the selected outcomes. Data on four of the bigger Empowered Action Group (EAG) states of India, namely Uttar Pradesh, Madhya Pradesh, Bihar, and Rajasthan, were obtained from the fourth round of the National Family Health Survey (NFHS-4) for analysis. A fixed-effect binary logistic regression model was used to assess the effect of FI teams and survey implementation design on the selected outcomes. To study the variation in the outcome variables at the interviewer level, a cross-classified multilevel model was used. Since one interviewer had worked in more than one primary sampling unit (PSU) & district and did not follow a perfect hierarchical structure, the cross-classified multilevel model was deemed suitable. In addition, since NFHS-4 used a two-stage stratified sampling design, two-level weights were adjusted for the models to compute unbiased estimates. This study demonstrated the presence of interviewer-level variation in the selected outcomes at both inter- and intra-field agencies across the selected states. The interviewer-level intra-class correlation coefficient (ICC) for women who had not availed antenatal care (ANC) was the highest for eastern Madhya Pradesh (0.23) and central Uttar Pradesh (0.20). For ‘immunisation card not seen’, Rajasthan (0.16) and western Uttar Pradesh (0.13) had higher interviewer-level ICC. Interviewer-level variations were insignificant for women who gave birth at home across all regions of Uttar Pradesh. Eastern Madhya Pradesh, Rajasthan, and Bihar showed higher interviewer-level variation across the selected outcomes, underlining the critical role of agencies and skilled interviewers in different survey implementation designs. The analysis highlights non-uniform adherence to survey protocols, which implies that not all interviewers and agencies performed in a similar manner in the field. This study recommends a refined mechanism for field implementation and supervision, including focused training on the challenges faced by FIs, random vigilance, and morale building. In addition, examining interviewer-level characteristics, field challenges, and field agency effects may also highlight the roots of interviewer-level variation in the data. However, based on the interviewer's performance in the field, the present study offers an intriguing insight into interviewer-level variations in the quality of data.

Keywords

• Survey design
• Survey implementation
• Interviewer effect
• Cross-classified multilevel model
• Level weights
• Maternal and child health