The effect of survey method on survey participation: Analysis of data from the Health Survey for England 2006 and the Boost Survey for London

Abstract

Abstract Background There is a need for local level health data for local government and health bodies, for health surveillance and planning and monitoring of policies and interventions. The Health Survey for England (HSE) is a nationally-representative survey of the English population living in private households, but sub-national analyses can be performed only at a regional level because of sample size. A boost of the HSE was commissioned to address the need for local level data in London but a different mode of data collection was used to maximise participant numbers for a given cost. This study examines the effects on survey and item response of the different survey modes. Methods Household and individual level data are collected in HSE primarily through interviews plus individual measures through a nurse visit. For the London Boost, brief household level data were collected through interviews and individual level data through a longer self-completion questionnaire left by the interviewer and collected later. Sampling and recruitment methods were identical, and both surveys were conducted by the same organisation. There was no nurse visit in the London Boost. Data were analysed to assess the effects of differential response rates, item non-response, and characteristics of respondents. Results Household response rates were higher in the 'Boost' (61%) than 'Core' (HSE participants in London) sample (58%), but the individual response rate was considerably higher in the Core (85%) than Boost (65%). There were few differences in participant characteristics between the Core and Boost samples, with the exception of ethnicity and educational qualifications. Item non-response was similar for both samples, except for educational level. Differences in ethnicity were corrected with non-response weights, but differences in educational qualifications persisted after non-response weights were applied. When item non-response was added to those reporting no qualification, participants' educational levels were similar in the two samples. Conclusion Although household response rates were similar, individual response rates were lower using the London Boost method. This may be due to features of London that are particularly associated with lower response rates for the self-completion element of the Boost method, such as the multi-lingual population. Nevertheless, statistical adjustments can overcome most of the demographic differences for analysis. Care must be taken when designing self-completion questionnaires to minimise item non-response.