The Effect of Program Design on Engagement With an Internet-Based Smoking Intervention: Randomized Factorial Trial

Abstract

BackgroundParticipant engagement influences treatment effectiveness, but it is unknown which intervention design features increase treatment engagement for online smoking cessation programs. ObjectiveWe explored the effects of 4 design features (ie, factors) on early engagement with an Internet-based, motivational smoking cessation program. MethodsSmokers (N=1865) were recruited from a large health care organization to participate in an online intervention study, regardless of their interest in quitting smoking. The program was intended to answer smokers’ questions about quitting in an effort to motivate and support cessation. Consistent with the screening phase in the multiphase optimization strategy (MOST), we used a 2-level, full-factorial design. Each person was randomized to 1 of 2 levels of each factor, including message tone (prescriptive vs motivational), navigation autonomy (dictated vs not), proactive email reminders (yes vs no), and inclusion of personally tailored testimonials (yes vs no). The effects of each factor level on program engagement during the first 2 months of enrollment were compared, including number of visits to the website resulting in intervention content views (as opposed to supplemental content views), number of intervention content areas viewed, number of intervention content pages viewed, and duration of time spent viewing this content, as applicable to each factor. ResultsAdjusting for baseline readiness to quit, persons who received content written in a prescriptive tone made the same number of visits to the website as persons receiving content in a motivational tone, but viewed 1.17 times as many content areas (95% CI 1.08-1.28; P<.001) and 1.15 times as many pages (95% CI 1.04-1.28; P=.009). Time spent viewing materials did not differ among groups (P=.06). Persons required to view content in a dictated order based on their initial readiness to quit made the same number of visits as people able to freely navigate the site, but viewed fewer content areas (ratio of means 0.80, 95% CI 0.74-0.87; P<.001), 1.17 times as many pages (95% CI 1.06-1.31; P=.003), and spent 1.37 times more minutes online (95% CI 1.17-1.59; P<.001). Persons receiving proactive email reminders made 1.20 times as many visits (95% CI 1.09-1.33; P<.001), viewed a similar number of content areas as persons receiving no reminders, viewed 1.58 times as many pages (95% CI 1.48-1.68; P<.001), and spent 1.51 times as many minutes online (95% CI 1.29-1.77; P<.001) as those who did not receive proactive emails. Tailored testimonials did not significantly affect engagement. ConclusionsUsing a prescriptive message tone, dictating content viewing order, and sending reminder emails each resulted in greater program engagement relative to the contrasting level of each experimental factor. The results require replication, but suggest that a more directive interaction style may be preferable for online cessation programs. Trial Registrationclinicaltrials.gov NCT00992264; http://clinicaltrials.gov/ct2/show/NCT00992264 (Archived by WebCite at http://www.webcitation.org/6F7H7lr3P)