Health and Social Care Delivery Research (Jul 2023)
Youth violence intervention programme for vulnerable young people attending emergency departments in London: a rapid evaluation
Abstract
Background Youth violence intervention programmes involving the embedding of youth workers in NHS emergency departments to help young people (broadly aged between 11 and 24 years) improve the quality of their lives following their attendance at an emergency department as a result of violent assault or associated trauma are increasing across the NHS. This study evaluates one such initiative run by the charity Redthread in partnership with a NHS trust. Objectives To evaluate the implementation and impact of a new youth violence intervention programme at University College London Hospital NHS Trust and delivered by the charity Redthread: (1) literature review of studies of hospital-based violent crime interventions; (2) evaluation of local implementation and of University College London Hospital staff and relevant local stakeholders concerning the intervention and its impact; (3) assessment of the feasibility of using routine secondary care data to evaluate the impact of the Redthread intervention; and (4) cost-effectiveness analysis of the Redthread intervention from the perspective of the NHS. Methods The evaluation was designed as a mixed-methods multiphased study, including an in-depth process evaluation case study and quantitative and economic analyses. The project was undertaken in different stages over two years, starting with desk-based research and an exploratory phase suitable for remote working while COVID-19 was affecting NHS services. A total of 22 semistructured interviews were conducted with staff at Redthread and University College London Hospital and others (e.g. a senior stakeholder involved in NHS youth violence prevention policy). We analysed Redthread documents, engaged with experts and conducted observations of staff meetings to gather more in-depth insights about the effectiveness of the intervention, the processes of implementation, staff perceptions and cost. We also undertook quantitative analyses to ascertain suitable measures of impact to inform stakeholders and future evaluations. Results Redthread’s service was viewed as a necessary intervention, which complemented clinical and other statutory services. It was well embedded in the paediatric emergency department and adolescent services but less so in the adult emergency department. The diverse reasons for individual referrals, the various routes by which young people were identified, and the mix of specific support interventions provided, together emphasised the complexity of this intervention, with consequent challenges in implementation and evaluation. Given the relative unit costs of Redthread and University College London Hospital’s inpatient services, it is estimated that the service would break even if around one-third of Redthread interventions resulted in at least one avoided emergency inpatient admission. This evaluation was unable to determine a feasible approach to measuring the quantitative impact of Redthread’s youth violence intervention programme but has reflected on data describing the service, including costs, and make recommendations to support future evaluation. Limitations The COVID-19 pandemic severely hampered the implementation of the Redthread service and the ability to evaluate it. The strongest options for analysis of effects and costs were not possible due to constraints of the consent process, problems in linking Redthread and University College London Hospital patient data and the relatively small numbers of young people having been engaged for longer-term support over the evaluation period. Conclusions We have been able to contribute to the qualitative evidence on the implementation of the youth violence intervention programme at University College London Hospital, showing, for example, that NHS staff viewed the service as an important and needed intervention. In the light of problems with routine patient data systems and linkages, we have also been able to reflect on data describing the service, including costs, and made recommendations to support future evaluation. Future work No future work is planned. Funding National Institute for Health and Care Research Health Services and Delivery Research programme (RSET: 16/138/17). Plain language summary Youth violence intervention programmes in the NHS embed specialist youth workers into a hospital’s paediatric emergency departments. These staff can engage young people and encourage positive change in their lives. Youth violence intervention programmes are part of a broader national strategy to prevent violence among young people. To improve our knowledge of the impact on young people and the cost-effectiveness of youth violence intervention programmes, we carried out an evaluation of a youth violence intervention programme introduced in 2020 at University College London Hospital and run by the charity Redthread. We reviewed the international evidence on youth violence intervention programmes, and other studies of Redthread services but found few studies measuring impact within the NHS. We reviewed documents and conducted 22 interviews with University College London Hospital and Redthread staff among others. We found that the service is viewed positively by NHS staff. We also found that youth workers can help a young person to better engage in their medical care and treatment. Youth violence intervention programmes also provide a link with non–health-care services within the community. Overall, they help NHS staff to better support vulnerable young people following discharge from hospital. We also established the cost of delivering Redthread services per user was £1865. This compares with a cost per inpatient of £5789 for a group of patients similar to those helped by Redthread. The average cost of a Redthread-type patient attending the emergency department was £203. We looked at whether it was possible to measure whether Redthread reduced young people’s re-admissions to the hospital’s emergency departments. However, we concluded that fully answering this question was not possible over the timescale of the project. This was because of the impact of COVID-19 on Redthread and other paediatric services, the low numbers of young people engaged in a longer-term programme with Redthread (59) and difficulties with linking information from the hospital and Redthread. We have therefore made various recommendations in this report to improve the way that data are collected and linked to aid future evaluations. Scientific summary Background Youth violence intervention programmes (YVIPs), and in particular those based in emergency departments (EDs), aimed broadly at young people aged between 11 and 24 years of age are part of a wider national strategy to tackle violence, the risk of violence or other types of harm. Despite many such programmes there is limited knowledge about their impact and cost-effectiveness. Prior attempts to demonstrate the efficacy of ED-based programmes have also been underpowered and results have been largely equivocal. Redthread has been implementing YVIPs in hospitals since 2006. In 2020, they started a service at University College London Hospitals NHS Trust (UCLH). This programme embeds specialist youth workers into some of the trust’s clinical departments, capitalising on ‘teachable moments’ to engage young people and encourage positive change in their lives. The aim of this study was to evaluate the implementation and impact of the Redthread intervention at UCLH with the following research questions: •What benefits does implementation of the Redthread YVIP have at UCLH for both staff and patients? •What evidence exists in the published research and grey literature about the effectiveness, benefits and impact of interventions in urgent care and hospital settings that focus on violent crime and young people? What lessons can be learned from UK and international studies to help NHS trusts implementing such interventions? •How can a combination of routine secondary care and Redthread data inform an evaluation of the impact of the Redthread service on the use of NHS hospital services? •What are the views of UCLH NHS staff on the Redthread intervention? •What organisational factors, processes, resources and staff training are necessary for the successful implementation and delivery of the Redthread service? •How cost-effective is the implementation of the Redthread service at UCLH? •What evaluation approaches and methodological designs appear particularly well suited and feasible for evaluations of the Redthread service and similar services in the NHS? Methods We undertook a mixed-methods evaluation in two phases. Phase 1 involved feasibility and scoping of the evaluation, including an exploratory search for published evidence. This was mostly undertaken while COVID-19 was affecting the service that Redthread was able to deliver at UCLH and was thus predominantly desk based. Activities were: •An exploratory review of the literature, including checking for other Redthread evaluations. •Nine semistructured interviews with Redthread staff, clinical staff involved with the early implementation of the service at UCLH (e.g. consultants working in paediatrics and children and young people’s services) and one senior NHS stakeholder involved in youth violence prevention programmes nationally. Interviews were analysed alongside Redthread documents to confirm Redthread’s programme theory, the intervention at UCLH and adaptations due to COVID-19. •An investigation into the feasibility of a quantitative evaluation of the service by studying local data flows and processes and analysing routine hospital data. •A desk-based review of available Redthread and UCLH documents to inform the economic analysis. Phase 2 (from April 2021, when the paediatric ED came back on site) involved a more in-depth study of the implementation at UCLH and other activities: •A targeted, scoping literature review to identify any recent published evidence. •A qualitative process evaluation involving 13 additional interviews with clinical and youth workers at UCLH and Redthread, plus three observations of three staff meetings, to understand the perceived impact and effectiveness of the service as well as identifying factors that enable the successful delivery of YVIPs. •Analysis of data collected by Redthread to understand more about the delivery of the service and those who engaged with it. •A cost–consequence analysis (CCA) using local data on the costs of the Redthread service and relevant hospital interventions. If we were able to establish during phase 1 that it would be possible to undertake a quantitative evaluation of the impact of service, then this would have been included in phase 2 alongside a cost-effectiveness analysis (CEA). However, we concluded that this was not going to be feasible and for an economic evaluation we adopted a CCA. Results Evidence review and current evaluation evidence (Chapter 3) We found a number of empirical studies, largely from North America, but limited peer-reviewed evidence from the UK for hospital-based interventions focused on young people. Available evidence indicates that young people who present in EDs from gunshot or knife- injuries, as well as other types of harm, are at significant risk of repeat injury. Moreover, young people are vulnerable to a variety of risks in the community and can therefore re-present to EDs because of physical assault, interpersonal violence, substance misuse and severe mental health problems. Because much of the existing empirical evidence comes from the United States, it is often associated with programmes that focus on gunshot or knife injuries as well as from other types of harm to young people (e.g. risky behaviour associated with drug and alcohol use). The impact of violence prevention programmes is mostly measured as hospital reattendance with reinjury and other measures such as service uptake, with many studies demonstrating that YVIP can be cost-effective and are often well received by young people. However, the range of youth-based interventions being studied has been wide, covering brief interventions to longer-term case work, and the quality of evidence is variable, with some studies including relatively small sample sizes and limited follow-up times. Of the 20 academic papers identified, only one specifically focused on the UK. We therefore found limited evidence of the impact of YVIPs within the NHS and UK health system, although Redthread has commissioned a number of independent evaluations at hospital level. There were few peer-reviewed studies applying qualitative research methods. Furthermore, there is a lack of randomised controlled trials and experimental studies specifically from the UK when compared with the United States. Overall, we found limited evidence about the impacts of these programmes on cohorts of young people from the UK, confirming conclusions from similar evidence reviews. Assessment of programme theory and implementation at UCLH (Chapter 4) Redthread interventions focus on young people aged 11 to 24 years who experience a traumatic event and present at hospital. Youth workers in the hospital EDs work alongside clinical staff to engage with victims of violence, assault or exploitation. Redthread’s programme theory draws on a number of influences, such as behaviour change theory and ‘contextual safeguarding’. The central concept is the ‘teachable moment’, which focuses on a youth worker initiating a dialogue with a young person about their health risks and their motivation and commitment to change. The Redthread service model at UCLH was consistent in terms of this programme theory, although the service had been adapted to local conditions and contingencies, for example on account of COVID-19. Within UCLH’s adolescent and paediatric services, the Redthread service was viewed positively by staff as filling a gap in service provision. Youth workers were able to help a young person to better engage in their medical care and treatment. They were also bridging non–health-care services within the community and thus enable front-line clinicians to better support vulnerable young people following discharge from hospital. Although there was severe disruption to the service due to the COVID-19 pandemic, by winter 2021/22 Redthread was perceived to be well embedded in the paediatric ED and adolescent services, and there was increasing awareness of the service in outpatient departments. Redthread and clinical staff noted that more could be done to raise staff awareness of Redthread across the trust, especially among nurses, junior doctors and other staff working in the adult ED. Identification of young people was not solely dependent on youth workers being in the ED. Other routes of referral included multidisciplinary team and safeguarding meetings, direct staff referrals, active searches of the hospital’s patient administration system (Epic) and the live board in ED. Reasons for referral were by no means limited to young people experiencing physical assault, but also included substance misuse, sexual assault, suicidal ideation and mental health crises. The key barriers to implementation included the impact of COVID-19, staff changes, lack of physical space for Redthread staff and difficulties engaging young people aged over 18 years presenting in the adult ED. Staff suggested various factors that helped embed the YVIP. These included championing by senior clinical staff, integration of Redthread staff in processes for identifying vulnerable young people, space near the ED to engage with young people, the ability for clinicians to refer young people via the hospital’s patient record system and clear and agreed operating procedures for the YVIP. Description and review of data used to manage the Redthread service at UCLH (Chapter 5) Redthread collect data on their service users; this enabled us to establish profiles of their characteristics. Some information is recorded on individuals who are referred but do not engage, which has, to some extent, enabled us to identify differences between the two groups and whether some types of individuals are more likely to engage than others. There is scope for improving the data captured by the local hospital system. For example, ensuring Redthread referrals are consistently flagged and, where recorded, indicating whether the individuals accepted or declined support. UCLH are improving their recording of information on the intent behind injuries, which would help the understanding of whether an individual was eligible to receive Redthread services. Costs of service at UCLH (Chapter 6) We were unable to conduct a CEA due to a lack of good-quality evidence describing the effect or impact on subsequent use of hospital services. A CCA showed that, over the course of a young person’s engagement with the service, a statistically significant decrease in some risks were found, specifically, for the ‘risks associated with experiencing further harm’, and with ‘not maintaining positive relationships with their families’. However, this analysis is limited by the small sample of patients and that these are subjective assessments of risk made by Redthread staff. The mean cost per Redthread user (for both those engaged in a longer-term programme and short-term crisis support combined) for the Redthread YVIP service over a 21-month period was calculated to be £1865. The mean cost for the emergency inpatient treatment of an artificially constructed group, similar to those likely to be referred to Redthread in UCLH, was estimated to be £5789, while the mean cost per attendance at the ED was £203. Feasibility of quantitative evaluation of service at UCLH (Chapter 7) A number of possible options for a rapid quantitative evaluation of the impact of the service on reducing hospital reattendance were considered, but it was concluded that none would be currently feasible due to: •Small numbers of young people who have so far engaged with the full longer-term Redthread programme at UCLH (59 over the period of the study). •Lack of consent to enable access to individual person-level data for Redthread users to link to hospital administrative data. •Likely difficulty in being able to detect the indirect impact of the service on wider groups of young people who live in the same neighbourhoods. •A lack of key information recorded in UCLH ED records. •Our inability to link national hospital inpatient and emergency care records due to the lack of linkable patient identifiers across the datasets. •The difficulty in identifying comparable control groups from routine hospital data. Matched case–control designs or approaches based on geographical areas of residence appeared to be the most feasible. We therefore made the following recommendations to facilitate future evaluation: •For analysts to use reattendance as one measure of impact, and work with clinicians and Redthread to develop criteria from routine hospital records that can be used to identify which attendances are potentially avoidable. •Similarly, develop criteria for identifying groups of patients attending other hospitals from which control groups can be selected. •For Redthread and acute hospital partners to consider mechanisms by which information from the service, as to who chooses to engage, and who chooses not to engage, can be linked to the routine data for analysis purposes. •For clinicians to routinely record when a Redthread referral has been made by using the relevant code in the patient administration system rather than only using free text. If a person is identified by Redthread themselves, then for this to be also flagged in the routine patient data. •That suitable geographical areas are decided upon between Redthread and UCLH, making it feasible to test an area-level approach. Limitations The implementation of the service was disrupted by the COVID-19 pandemic, which affected how the service developed within UCLH. It also made it difficult to engage with staff outside Redthread and hospital paediatric services, in particular those working within the adult ED. We were unable to conduct any observations on site and qualitative data were collected remotely. We were also unable to approach young people who had engaged with Redthread for a number of reasons outlined in this report (e.g. patient confidentiality, the sensitive nature of the clinical cases presenting at UCLH). Further work Suggested areas of further work include studies of the perceptions and experiences of young people receiving Redthread support, studies that are able to overcome the data challenges we have encountered in our own study, studies that look beyond hospital-based outcomes and multisite case studies. Conclusions Our evaluation was not able to determine a feasible approach to measuring the quantitative impact of Redthread’s YVIP at UCLH in the time available, but we have been able to reflect on data describing the service, including costs, and make recommendations to support future evaluation. We have been able to contribute to the qualitative evidence on the implementation of the service. Redthread’s service was largely viewed as a necessary service for young people at risk of harm (beyond involvement in violence), and one which was complementary to clinical and other statutory services. The service became particularly well embedded in paediatric ED and adolescent services, but less so in the adult ED, possibly in part as a consequence of the impact of COVID-19. The diverse reasons behind individual referrals, the various routes by which young people were identified, and the mix of specific support interventions provided together emphasised the view that this was a complex intervention, with challenges in implementation. Recently published guidance to support implementation of violence reduction services has emphasised the need for evaluation to be undertaken as a key improvement activity and touches on data that ought to be routinely collected. There is a clear need for good quality evidence of impact and our recommendations may help to improve future evaluation. Funding National Institute for Health and Care Research Health Services and Delivery Research programme (RSET: 16/138/17).
Keywords