Impact of sampling and data collection methods on maternity survey response: a randomised controlled trial of paper and push-to-web surveys and a concurrent social media survey

Abstract

Abstract Background Novel survey methods are needed to tackle declining response rates. The 2020 National Maternity Survey included a randomised controlled trial (RCT) and social media survey to compare different combinations of sampling and data collection methods with respect to: response rate, respondent representativeness, prevalence estimates of maternity indicators and cost. Methods A two-armed parallel RCT and concurrent social media survey were conducted. Women in the RCT were sampled from ONS birth registrations and randomised to either a paper or push-to-web survey. Women in the social media survey self-selected through online adverts. The primary outcome was response rate in the paper and push-to-web surveys. In all surveys, respondent representativeness was assessed by comparing distributions of sociodemographic characteristics in respondents with those of the target population. External validity of prevalence estimates of maternity indicators was assessed by comparing weighted survey estimates with estimates from national routine data. Cost was also compared across surveys. Results The response rate was higher in the paper survey (n = 2,446) compared to the push-to-web survey (n = 2,165)(30.6% versus 27.1%, difference = 3.5%, 95%CI = 2.1–4.9, p < 0.0001). Compared to the target population, respondents in all surveys were less likely to be aged < 25 years, of Black or Minority ethnicity, born outside the UK, living in disadvantaged areas, living without a partner and multiparous. Women in the social media survey (n = 1,316) were less representative of the target population compared to women in the paper and push-to-web surveys. For some maternity indicators, weighted survey estimates were close to estimates from routine data, for other indicators there were discrepancies; no survey demonstrated consistently higher external validity than the other two surveys. Compared to the paper survey, the cost saving per respondent was £5.45 for the push-to-web survey and £22.42 for the social media survey. Conclusions Push-to-web surveys may cost less than paper surveys but do not necessarily result in higher response rates. Social media surveys cost significantly less than paper and push-to-web surveys, but sample size may be limited by eligibility criteria and recruitment window and respondents may be less representative of the target population. However, reduced representativeness does not necessarily introduce more bias in weighted survey estimates.

Keywords

• Survey
• Questionnaire
• Sampling
• Response rate
• Representativeness
• Non-response bias