Health and Social Care Delivery Research (Nov 2023)
The effectiveness and acceptability of multimedia information when recruiting children and young people to trials: the TRECA meta-analysis of SWATs
Abstract
Background The information provided to potential trial participants plays a crucial role in their decision-making. Printed participant information sheets for trials have received recurrent criticism as being too long and technical, unappealing and hard to navigate. An alternative is to provide information through multimedia (text, animations, video, audio, diagrams and photos). However, there is limited evidence on the effects of multimedia participant information on research recruitment rates, particularly in children and young people. Objectives The study objectives were as follows: to develop template multimedia information resources through participatory design, for use when recruiting children and young people to trials to evaluate the multimedia information resources in a series of Studies Within A Trial, to test their effects on recruitment and retention rates, and participant decision-making, by comparing the provision of multimedia information resources instead of printed participant information sheets, and comparing the provision of multimedia information resources in addition to printed participant information sheets. Design Two-phase study: multimedia information resources development including qualitative study; user testing study; readability metrics; enhanced patient and public involvement multimedia information resources’ evaluation comprising Studies Within A Trial undertaken within host trials recruiting children and young people. Setting United Kingdom trials involving patients aged under 18. Participants Development phase: n = 120 (children and young people, parents, clinicians, trial personnel). Evaluation phase: n = 1906 (children and young people being asked to take part in trials). Interventions Multimedia information resources (comprising text, audio, ‘talking heads’ video, trial-specific and trial-generic animations). Printed participant information sheets. Main outcome measures Primary outcome: trial recruitment rate comparing multimedia information resource-only with printed participant information sheet-only provision. Secondary outcomes: trial recruitment rate comparing combined multimedia information resource and printed participant information sheet with printed participant information sheet-only provision; trial retention rate; quality of participant decision-making. Results for each trial were calculated and combined in a two-stage random-effects meta-analysis. Results Phase 1 generated two multimedia information resource templates: (1) for children aged 6–11 years; (2) for children aged 12–18 years and parents. In the Phase 2 Studies Within A Trial the multimedia information resources improved trial recruitment, when compared to printed information alone [odds ratio (OR) = 1.54; 95% confidence interval (CI) 1.05 to 2.28; p = 0.03; I2 = 0%]. When printed participant information sheet-only provision was compared to combined multimedia information resource and printed participant information sheet provision, there was no effect on trial recruitment (OR = 0.89; 95% CI 0.53 to 1.50; I2 = 0%). There were no differences between multimedia information resource and printed participant information sheet on trial retention or participant decision-making quality. In a study within a hypothetical trial setting, multimedia information resource-only provision produced higher ratings of ‘information was easy to understand’ (Z = 3.03; p = 0.003) and ‘I had confidence in decision-making’ (Z = 2.00; p = 0.044) than printed participant information sheet-only provision. Limitations It was not possible to include data from three Studies Within A Trial in the meta-analysis due to limited sample size, and questionnaire return rates were low, which reduced the strength of the findings. Conclusions Use of multimedia information increased the rate of recruitment to trials involving children and young people compared to standard patient information sheets. Future work There should be further evaluation of the effects of multimedia information on recruitment to trials involving children and young people. It would be valuable to assess any impacts of multimedia information resources on communication between trial recruiters, children and young people, and parents. Study registration This trial is registered as TRECA ISRCTN 73136092 and Northern Ireland Hub for Trials Methodology Research SWAT Repository (SWAT 97). Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 14/21/21) and is published in full in Health and Social Care Delivery Research; Vol. 11, No. 24. See the NIHR Funding and Awards website for further award information. Plain language summary Clinical trials are important to National Health Service care, but it can be difficult to recruit enough people. We do not know enough about how to improve recruitment, especially when trying to recruit children and young people. People are normally told about a trial through printed information, which is often long and complex. Multimedia information (text, audio, cartoons and video) might be a better way of telling people. It is important to test whether multimedia interventions can help. One way of doing this is to run a ‘Study Within A Trial’ where people receive information in different ways. We created two multimedia interventions, one for parents and young people being asked for consent, and a simpler one for younger children. Some content applied to all trials, and some about the specific trial people were being asked to consider. We designed these by working closely with children and young people, parents and healthcare staff. We tested the multimedia information in six trials (although only three gave us enough data). Children, young people and their parents saw either standard printed information or our multimedia information. We then collected data on their decision-making, trial recruitment and whether people stayed in the trial. Children and young people who saw multimedia information were more likely to be recruited than those who received standard printed information. Once recruited to a trial, people given multimedia or printed information were similarly likely to remain in the trial. People’s views on multimedia and printed information were also similar, but this finding could have been affected by small numbers of people returning questionnaires. Our study provides evidence that multimedia information can be used in trials with children and young people and that it increases the number of people who agree to take part, but further work is needed. Scientific summary Background There is insufficient evidence on how we should treat children and young people across a range of health conditions, in part due to a lack of trial evidence. The information provided to potential trial participants plays a crucial role in their decision-making about participation. In most cases, mandated recruitment information is provided to patients as printed participant information sheets, which have received recurrent criticism as being too long and technical, unappealing and hard to navigate. An alternative is to provide information digitally, enabling the use of multimedia information in trial recruitment, that is, through animations, video, audio, diagrams and photos, as well as written text. There is limited evidence on the effects of multimedia participant information on research recruitment rates, particularly in children and young people. Objectives The study objectives were as follows: to develop template multimedia information resources through participatory design, for use when recruiting children and young people to trials to obtain and analyse qualitative data from interviews and focus groups with members of key stakeholder groups (i.e. young patients; parents or guardians; and clinicians) to ensure the content, format and delivery of the multimedia information resources reflect their preferences to user-test the multimedia information resources with children and young people (and parents/guardians), to test the ability of the multimedia information resources to inform potential users to establish a Patient and Parent Advisory Group and actively liaise with them in the development and evaluation of the multimedia information resources to evaluate the multimedia information resources in a series of Studies Within A Trial, to test their effects on recruitment and retention rates, and participant decision-making, by comparing the provision of multimedia information resources instead of printed participant information, and the provision of multimedia information resources in addition to printed participant information. Methods The study involved a two-phase design: development of the multimedia information resources, followed by evaluation. The Phase 1 development included the following four elements: A qualitative study with stakeholder groups, who participated across a two-round study. In the first round, participants discussed information priorities in relation to trial recruitment, and preferences for multimedia information resource appearance and content. In the second round, participants commented on a prototype multimedia information resource for a trial. Data collection used individual or joint interviews, and focus discussion groups. Data analysis was initially deductive descriptive, to rapidly inform the multimedia information resource development; subsequent analyses were iterative and thematic. A user testing study involved performance-based testing on two versions of the prototype trial multimedia information resource, one for children aged 6–11 and the other for children aged 12–18 and parents. Participants used one of two prototype multimedia information resources to answer ten factual questions on its content. For each question participants had to find the relevant information in the multimedia information resource and, once found, show their understanding. Amending the written text within the multimedia information resources. Enhanced patient and parent involvement. The Phase 2 evaluation phase comprised a series of Studies Within A Trial within six host trials recruiting children and young people. Participants were randomised to receive trial recruitment information in one of three formats: multimedia information resource-only; participant information sheet-only; or combined multimedia information resource and participant information sheet. The primary outcome was trial recruitment in the multimedia information resource-only versus the participant information sheet-only arms. Secondary outcomes were as follows: trial retention; participant decision-making (assessed by Likert scale); and trial recruitment, trial retention and participant decision-making when comparing combined multimedia information resource and participant information sheet with participant information sheet-only. Data from the Studies Within A Trial were combined in a pre-planned statistical meta-analysis. We also undertook a trial in a hypothetical trial setting, comparing the effects on multimedia information resource-only and participant information sheet-only provision on decision-making quality. Results Qualitative study Participants were 62 individuals (21 children and young with long-term health conditions, aged 6–19; 24 parents; 17 clinicians or researchers) who took part across a two-round study. Participants emphasised the need for the views of children and young people to be central to multimedia information resource design. Content seen as important included the following: that leaving a trial was ok; what participation would involve; ensuring that multimedia information resource imagery was not ‘scary’; what the trial was testing; potential risks associated with the trial; possible benefits of participation; confidentiality. Sources of trial funding, and payments to participants were seen as less important. They wanted multimedia information resources to be colourful but also appear ‘professional’. They selected from a choice of animation character sets. They had strong preferences for: plain fonts; clear multimedia information resource structure; concise animations with engaging voiceover. A strong theme was being able to identify with participation, possibly through use of video testimony from trial leaders, participants and parents. There were mixed views on interactivity within the multimedia information resources. User testing The first round of testing involved 26 participants (seven children aged 7–11 with a parent; six adolescents aged 12–17; seven parents of adolescents). Ninety-two per cent of information was found without difficulty, and comprehension was high (85% answered all correct). Following minor revisions to wording and layout of the multimedia information resources, they were tested in round 2 with 26 new participants (seven children aged 7–11 with a parent; six adolescents aged 12–17; seven parents of adolescents). Ninety-three per cent of information was found without difficulty, and comprehension was high (95% answered all correct). Following the second user testing round, changes were made to layout of multimedia information resource ‘tabs’ and further information was added on potential risks and contacting trial personnel. Amending the written text within the multimedia information resources involved using the approved version of the trial participant information sheet and rewriting (through plain English and readability metrics, to ensure it was age-appropriate) and restructuring (through reduced number of headings, to aid navigation). Enhanced patient and parent involvement through the formation of a Patient and Parent Advisory Group comprising three people aged 19–24 with long-term health conditions, and three parents of young people with long-term health conditions. The Patient and Parent Advisory Group met three-monthly throughout the study and, between meetings, provided feedback on planned methods, multimedia information resource written content, and animation characters and storyboards. The Phase 1 development phase informed two template multimedia information resources (one for children and young people aged 6–11; one for children and young people aged 12–18 and parents), which would include all written text of the trial participant information sheet, short ‘talking head’ videos with trial personnel and a participant, and five short, animated videos with voiceover (an explainer, which was trial-specific; and four that were trial-generic, explaining aspects of trials). SWAT evaluation The Phase 2 evaluation phase comprised six Studies Within A Trial within different host trials, plus a small randomised controlled trial with adolescents asked to imagine themselves being recruited to a trial. In the Study Within A Trial potential host trial participants were randomly allocated to receive recruitment information as follows: participant information sheet-only; multimedia information resource-only; or combined multimedia information resource and participant information sheet. The multimedia information resources were accessed on PC, laptop, tablet computer or phone. The six host trials were recruiting children and young people with a range of ages and health conditions. Three of the Studies Within A Trial generated sufficient data for logistic regression models, which were combined within a preplanned statistical meta-analysis. Unfortunately, three Studies Within A Trial could not be used to generate logistic regression models, due to being too small (n = 2) or having insufficient variation in outcomes (n = 1). Meta-analysis was based on three Studies Within A Trial (total n = 1758). Providing potential trial participants with multimedia information resource rather than participant information sheet resulted in an increase in trial recruitment (OR = 1.54; 95% CI 1.05 to 2.28; p = 0.03; I2 = 0%), which was statistically significant. It resulted in no increase in rates of retention in trials (OR = 1.29; 95% CI 0.36 to 4.65; p = 0.70; I2 = 0%), assessed at 6 weeks to 6 months after randomisation in the host trial. Decision-Making Questionnaire scores were the same in the multimedia information resource-only and participant information sheet-only arms: adjusted mean difference –0.79 (95% CI –2.80 to 1.22; p = 0.44; I2 = 53.6%). Providing potential trial participants with combined multimedia information resource and participant information sheet rather than participant information sheet-only did not result in an increase in trial recruitment (OR = 0.89; 95% CI 0.53 to 1.50; p = 0.67; I2 = 0%) or trial retention (OR = 2.18; 95% CI 0.48 to 10.00; p = 0.31; I2 = 0%). Providing combined multimedia information resource and participant information sheet resulted in lower Decision-Making Questionnaire scores (OR = –2.07; 95% CI –4.13 to –0.01; p = 0.05; I2 = 0%), which was borderline statistically significant. Rates of Decision-Making Questionnaire scores in the Study Within A Trial was low, particularly among those who declined host trial recruitment. In the trial undertaken in a hypothetical setting, multimedia information resource-only provision produced higher ratings of ‘information was easy to understand’ (Z = 3.03; p = 0.003) and ‘I had confidence in decision-making’ (Z = 2.00; p = 0.044) than participant information sheet-only provision, with no differences between arms on the other seven questionnaire items. Conclusions The study produced two multimedia information resource templates (one for younger children aged 6–11; and one for older children aged 12–18 and parents), which were informed by significant empirical co-design work to ensure they met the preferences and information needs of potential trial participants and those recruiting them. The qualitative study found that children and young people, and their parents, said they would place high value on knowing about the experiences of trial participation, when making consent decisions themselves. Trials should consider including short video, audio or written accounts from trial personnel and current trial participants in their recruitment information. The multimedia information resources were acceptable to host trials and Research Ethics Committees, and there was no evidence of resistance to this format of recruitment information. Six Studies Within A Trial were undertaken with host trials recruiting children and young people; three were successful in providing data but three were not successful due to funding, governance and COVID-19 challenges to recruitment. Providing multimedia trial information rather than standard participant information sheets increased trial recruitment rates, which could reduce trial recruitment periods. Trials should consider using multimedia information as an alternative to print, although further evaluation is needed before it can be recommended for routine use. Providing multimedia trial information rather than standard participant information sheets did not increase trial retention or quality of participant decision-making. Providing multimedia trial information in addition to standard participant information sheets did not increase trial recruitment or retention, and resulted in lesser quality of participant decision-making, and it does not appear to be beneficial for trials to develop both formats of information for concurrent provision to potential participants, although further evidence is needed to test this conclusion. Further evaluation of the use of multimedia information in trial recruitment involving other population groups (including trials involving adults) is needed, as the current evidence base is lacking. There is a need for research to evaluate which elements of multimedia information are used by potential trial participants when making consent decisions. There is a need for research to evaluate the impact of multimedia information on communication between potential trial participants and those involved in recruiting them. Research studies using novel information formats in research recruitment or retention processes should ideally use a form of co-design and/or user testing before implementing the information. Study registration This trial is registered as TRECA ISRCTN 73136092 and Northern Ireland Hub for Trials Methodology Research SWAT Repository (SWAT 97). Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 14/21/21) and is published in full in Health and Social Care Delivery Research; Vol. 11, No. 24. See the NIHR Funding and Awards website for further award information.
Keywords
