Ask (Jan 2017)
A Meta-analysis of Within-Household Selection Impact on Survey Outcome Rates, Demographic Representation and Sample Quality in the European Social Survey
Abstract
Face-to-face surveys of the general population often start with a probability sample of the households or addresses. However, even if a probability sample of households or addresses has been drawn, random selection of the target respondent within the selected household is crucial for obtaining a probability sample of individuals comprising the population. Over a dozen procedures of within-household selection have been described in survey literature. This article is concerned only with the two most popular of these, i.e., with the Kish grid procedure and the class of birthday procedures (i.e., next-birthday, last-birthday and closest-birthday methods). The main goal of this paper is to address the question of whether the Kish grid and birthday methods differ in their impact on: (1) refusal and cooperation rates, and (2) demographic representation of the survey sample, as well as (3) the degree of interviewers' influence on the selection process. Based on 98 different surveys from all seven rounds of the European Social Survey, a meta-analysis was conducted to generate quantitative measures indicating the size of the overall impact of Kish grid and birthday procedures. Several conclusions can be formulated based on the analysis. Firstly, Kish grid samples (compared to birthday samples) result in significantly higher odds of receiving refusals and significantly lower odds of obtaining cooperation. Secondly, both Kish grid and birthday samples have a similar and significant impact on gender and age imbalance. Finally, birthday procedures give interviewers greater opportunity to influence the selection process. The latter means that the use of Kish grid samples is usually associated with a higher quality of the within-household selection, even though in such samples refusal rates are significantly higher and cooperation rates are significantly lower compared to those obtained in birthday samples.
Keywords
