The Influence of Response Mode on Study Results: Offering Cigarette Smokers a Choice of Postal or Online Completion of a Survey

Abstract

BackgroundIt is unclear whether offering online data collection to study participants affects compliance or produces bias. ObjectiveTo compare response rates, baseline characteristics, test-retest reliability, and outcomes between cigarette smokers who chose to complete a survey by mail versus those who chose to complete it online. MethodsWe surveyed cigarette smokers who intended to stop smoking within the next 30 days to determine barriers to calling a smoking quit line. Participants were offered the choice of completing a paper version of the survey sent through the mail or an online version at a password-protected website. Participants were called 2 months later to determine if they had made a quit attempt and/or called a smoking quit line since the baseline survey. We compared characteristics and outcomes among those who chose postal versus online completion. We measured test-retest reliability of the baseline survey by resurveying a semirandom sample of participants within 10 days of the original survey. ResultsOf 697 eligible respondents to newspaper ads in 12 US cities, 438 (63%) chose to receive a mailed paper survey and 259 (37%) chose an Internet survey. Survey return rates were the same for the 2 modes (92% versus 92%, P = .82). Online respondents were younger (mean of 46 versus 51 years old for postal, P < .001), more likely to be white (76% versus 62%, P < .001), less likely to be African American (18% versus 30%, P < .001), more highly educated (34% college graduate versus 23%, P < .001), more likely to intend to stop smoking in the next 30 days (47% definitely versus 30%, P < .001), and more likely to have heard of a smoking quit line (51% versus 40%, P = .008). Participants did not differ on gender (54% female for online versus 55% for postal, P = .72) or cigarettes smoked per day (mean of 19 versus 21, P = .30). Online respondents had slightly fewer missing items on the 79-item survey (mean of 1.7% missing versus 2.3%, P = .02). Loss to follow-up at 2 months was similar (16% for online and 15% for postal, P = .74). There was no significant difference between online and postal respondents in having called a smoking quit line during the 2-month follow-up period (20% versus 24%, P = .22) or in having made a quit attempt (76% versus 79%, P = .41). ConclusionsCigarette smokers who chose to complete a survey using the Internet differed in several ways from those who chose mailed surveys. However, more importantly, online and postal responses produced similar outcomes.