Revisiting "yes/no" versus "check all that apply": Results from a mixed modes experiment

Abstract

The work of Smyth, Dillman, Christian, and Stern (2006) and Smyth, Christian, and Dillman (2008) compares “yes/no” questions to “check all that apply” questions. They conclude that the “yes/no” format is preferable as it reflects deeper processing of survey questions. Smyth et al. (2008) found that the “yes/no” format performed similarly across telephone and web modes. In this paper we replicate their research and extend it by including a comparison with face-to-face in addition to telephone and web and by using probability samples of the general adult population. A cognitive interviewing follow-up was used to explore the quantitative findings. Our results suggest there are times when the “yes/no” format may not perform similarly across modes and that there may be factors which limit the quality of answers.

Keywords

• survey methods
• mode of data collection
• questionnaire design
• check all that apply
• forced choice