Accounting for Patient Engagement in Randomized Controlled Trials Evaluating Digital Cognitive Behavioral Therapies

Abstract

Background: Cognitive behavioral therapy (CBT) can be a useful treatment option for various mental health disorders. Modern advances in information technology and mobile communication enable delivery of state-of-the-art CBT programs via smartphones, either as stand-alone or as an adjunct treatment augmenting traditional sessions with a therapist. Experimental CBTs require careful assessment in randomized clinical trials (RCTs). Methods: We investigate some statistical issues for an RCT comparing efficacy of an experimental CBT intervention for a mental health disorder against the control. Assuming a linear model for the clinical outcome and patient engagement as an influential covariate, we investigate two common statistical approaches to inference—analysis of covariance (ANCOVA) and a two-sample t-test. We also study sample size requirements for the described experimental setting. Results: Both ANCOVA and a two-sample t-test are appropriate for the inference on treatment difference at the average observed level of engagement. However, ANCOVA produces estimates with lower variance and may be more powerful. Furthermore, unlike the t-test, ANCOVA allows one to perform treatment comparison at the levels of engagement other than the average level observed in the study. Larger sample sizes may be required to ensure experiments are sufficiently powered if one is interested in comparing treatment effects for different levels of engagement. Conclusions: ANCOVA with proper adjustment for engagement should be used for the for the described experimental setting. Uncertainty on engagement patterns should be taken into account at the study design stage.

Keywords

• analysis of covariance
• digital cognitive behavioral therapy
• engagement as a covariate
• randomized controlled trial
• sample size planning