Internet Interventions (Dec 2023)
A chatbot-delivered intervention for optimizing social media use and reducing perceived isolation among rural-living LGBTQ+ youth: Development, acceptability, usability, satisfaction, and utility
Abstract
Background: Lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ+) youth are at higher risk of isolation and depression than their heterosexual peers. Having access to tailored mental health resources is a documented concern for rural living LGBTQ+ youth. Social media provides access to connections to a broader and like-minded community of peers, but it also is a vehicle for negative interactions. We developed REALbot, an automated, social media–based educational intervention to improve social media efficacy, reduce perceived isolation, and bolster connections for rural living LGBTQ+ youth. This report presents data on the acceptability, feasibility, and utility of REALbot among its target audience of rural living LGBTQ+ youth. Methods: We conducted a week-long exploratory study with a single non-comparison group of 20 rural-living LGBTQ+ youth aged 14–19 recruited from social media to test our Facebook- and Instagram-delivered chatbot. We assessed pre- and post-test scores of social media self-efficacy, social isolation (4-item Patient-Reported Outcomes Measurement System – PROMIS), and depressive symptoms (Patient Health Questionnaire, Adolescent Version – PHQ-A). At post-test, we assessed acceptability (User Experience Questionnaire – UEQS), usability (Chatbot Usability Questionnaire –CUQ and Post-Study Satisfaction and Usability Questionnaire –PSSUQ), and satisfaction with the chatbot (Client Satisfaction Questionnaire – CSQ), along with two open-ended questions on ‘likes’ and ‘dislikes’ about the intervention. We compared pre- and post-test scores with standard univariate statistics. Means and standard deviations were calculated for usability, acceptability, and satisfaction. To analyze the responses to post-test open-end questions, we used a content analysis approach. Results: Acceptability of REALbot was high with UEQ-S 5.3 out of 7 (SD = 1.1) and received high usability scores with CUQ and PSSUQ (mean score (M) = 78.0, SD = 14.5 and M = 86.9, SD = 25.2, respectively), as well as high user satisfaction with CSQ (M = 24.9, SD = 5.4). Themes related to user ‘likes’ and ‘dislikes’ were organized in two main categories: usability and content provided. Participants were engaged with the chatbot, sending an average of 49.3 messages (SD = 43.6, median = 30). Pre-/post- changes in scores of perceived isolation, depressive symptoms and social media self-efficacy were not significant (p's > 0.08). Conclusion: REALbot deployment was found to be feasible and acceptable, with good usability and user satisfaction scores. Participants reported changes from pre- to post-test in most outcomes of interest and effect sizes were small to medium. Additional development and a formal evaluation of feasibility and engagement with behavioral targets is warranted.
