Health and Social Care Delivery Research (Jun 2023)
Early evaluation of the Children and Young People’s Mental Health Trailblazer programme: a rapid mixed-methods study
Abstract
Background The Children and Young People’s Mental Health Trailblazer programme is funding the creation of new mental health support teams to work in schools and further education colleges. Mental health support teams directly support children and young people with ‘mild to moderate’ mental health problems and work with school and college staff to promote well-being for all. A new workforce of education mental health practitioners is being trained for the teams. Objective(s) The National Institute for Health and Care Research Birmingham, RAND and Cambridge Evaluation Rapid Evaluation Centre and Policy Innovation and Evaluation Research Unit undertook an early evaluation of the Trailblazer programme to examine the development, implementation and early progress of mental health support teams in the programme’s first 25 ‘Trailblazer’ sites. Design A mixed-methods evaluation, comprising three work packages: 1.Establishing the baseline and understanding the development and early impacts of the Trailblazer sites, including two rounds of surveys with key informants and participating education settings in all 25 sites. 2.More detailed research in five purposively selected Trailblazer sites, including interviews with a range of stakeholders and focus groups with children and young people. 3.Scoping and developing options for a longer-term assessment of the programme’s outcomes and impacts. Fieldwork was undertaken between November 2020 and February 2022. The University of Birmingham Institute for Mental Health Youth Advisory Group was involved throughout the study, including co-producing the focus groups with children and young people. Results Substantial progress had been made implementing the programme, in challenging circumstances, and there was optimism about what it had the potential to achieve. The education mental health practitioner role had proven popular, but sites reported challenges in retaining education mental health practitioners, and turnover left mental health support teams short-staffed and needing to re-recruit. Education settings welcomed additional mental health support and reported positive early outcomes, including staff feeling more confident and having faster access to advice about mental health issues. At the same time, there were concerns about children who had mental health problems that were more serious than ‘mild to moderate’ but not serious enough to be accepted for specialist help, and that the interventions offered were not working well for some young people. Mental health support teams were generally spending more time supporting children with mental health problems than working with education settings to develop ‘whole school’ approaches to mental health and well-being, and service models in some sites appeared to be more clinically oriented, with a strong focus on mental health support teams’ therapeutic functions. Limitations Despite efforts to maximise participation, survey response rates were relatively low and some groups were less well represented than others. We were not able to gather sufficiently detailed data to develop a typology of Trailblazer sites, as was planned. Conclusions Key lessons for future programme implementation include: –Whether mental health support teams should expand support to children and young people with more complex and serious mental health problems. –How to keep the twin aims of prevention and early intervention in balance. –How to retain education mental health practitioners once trained. Future work The findings have important implications for the design of a longer-term impact evaluation of the programme, which is due to commence in summer 2023. Study registration Ethical approval from the University of Birmingham (ERN_19-1400 – RG_19-190) and London School of Hygiene and Tropical Medicine (Ref: 18040) and Health Research Authority approval (IRAS 270760). Funding The Birmingham, RAND and Cambridge Evaluation Rapid Evaluation Centre is funded by the National Institute for Health and Care Research Health Services and Delivery Research programme (HSDR 16/138/31). The Policy Innovation and Evaluation Research Unit is funded by the NIHR Policy Research Programme (PR-PRU-1217-20602). A note on terminology This report uses the term ‘children and young people’s mental health services’ to describe all services that support children and young people who have difficulties with their mental health and emotional well-being. These services encompass prevention and universal provision, through to specialist and crisis support, including inpatient care. They are provided by NHS, local authority, voluntary, community and independent sector services, as well as schools and colleges. Some participants in the study refer to ‘child and adolescent mental health services’ (or CAMHS), which is an older term for specialist NHS mental health services for young people aged 0–18 (or, in some areas, 0–25) years. We also use the term ‘whole school approach’ to describe all the ways in which schools and colleges can address the emotional health of children and/or young people in their setting, which includes supporting those who are experiencing mental health problems to access appropriate help. In wider literature and debate, these activities are also referred to as holistic, universal, graduated or school-wide approaches to mental health and well-being. The national programme launched by the Department of Health and Department for Education to implement the proposals in the 2017 Green Paper Transforming Children and Young People’s Mental Health Provision was originally termed the Trailblazer programme. It was subsequently renamed the Children and Young People’s Mental Health Implementation programme, and only the first wave of sites was referred to as Trailblazers. As the study reported herein focuses exclusively on this first wave of Trailblazer sites, we have opted to use the programme’s original name. Plain language summary The Children and Young People’s Mental Health Trailblazer programme started in 2018 and is funding the creation of new mental health support teams to work in schools and further education colleges. Mental health support teams directly support children and young people with ‘mild to moderate’ mental health problems and help schools and colleges to promote well-being for all. The programme is also creating and training a new workforce of education mental health practitioners. Our study looked at the experiences of setting up and running mental health support teams in the first 25 areas involved in the programme (called Trailblazers). We wanted to understand what the teams were doing day-to-day, who was working in them, what was going well, whether there were any challenges, and what progress they were making. To do this, we looked at documents and information provided by the national programme team and collected our own data using surveys, interviews and focus groups. We found that the programme was making good progress, and that schools and colleges welcomed having additional mental health support. Children and young people told us how important it was to have somebody in their school or college who they could speak to about their mental health. Mental health support teams were generally spending more time supporting young people who had mental health difficulties than working with education settings to promote emotional well-being across the whole school or college community. There were also challenges. Some children had mental health problems that were more serious than ‘mild to moderate’ but not serious enough to be accepted for specialist support. The type of support that mental health support teams were providing was not suitable for all children and young people. Once education mental health practitioners had been trained, some moved on from their role quite quickly, leaving teams short-staffed. The findings have important implications for the design of a longer-term study to assess the impact of the programme, due to commence in summer 2023. Scientific summary Background The Children and Young People’s Mental Health Trailblazer programme was launched in 2018 to take forward the proposals set out in the Transforming Children and Young People’s Mental Health Provision Green Paper. The programme is being implemented in successive waves, with the first wave funding the creation of 58 mental health support teams (MHSTs) in 25 ‘Trailblazer’ sites. Across these sites, 1050 schools and further education colleges were recruited to participate in the programme, each of which received support from an MHST and was encouraged to appoint a senior lead for mental health for their setting (if they did not already have one in place). MHSTs have three core functions: (1) providing direct support to children and young people with mild to moderate mental health issues; (2) supporting education settings to introduce or develop their whole school/college approach to mental health and well-being; and (3) giving advice to staff in education settings and liaising with external specialist services to help children and young people to get the right support and stay in education. A new professional role has been created for the programme: education mental health practitioner (EMHP). The programme is being implemented in the context of a children’s mental health service under strain. Considerable and increasing levels of mental ill health in children and young people, historic underinvestment in children’s mental health services and the COVID-19 pandemic have contributed to services struggling to cope with increasing demand. Objectives The National Institute for Health and Care Research (NIHR) Birmingham, RAND and Cambridge Evaluation Rapid Evaluation Centre and Policy Innovation and Evaluation Research Unit undertook an early, process-oriented evaluation of the Trailblazer programme to examine the development, implementation and early progress of the MHSTs in the Trailblazer sites. The aims of the evaluation were to: 1.Understand the baseline position and contextual features of the Trailblazer sites, including the accessibility, quality and effectiveness of existing mental health services and support in education settings and perceived gaps in provision prior to the programme commencing. 2.Describe and understand the emerging delivery models, their leadership and governance, and explore how these vary across the Trailblazer sites and the potential implications of this variation for future effectiveness of the programme. 3.Describe the experience of MHSTs, education settings, clinical commissioning groups and local authority commissioners, children and young people’s mental health services and others of taking part in the delivery of the programme. 4.Capture views about the progress being made by Trailblazers towards the goals of the programme, early impacts and any unanticipated consequences in the initial phases of the programme. 5.Identify measures and data sources of relevance to assessing programme outcomes and costs as well as appropriate comparator areas and education settings to assess the feasibility and develop the design of a long-term outcome and economic evaluation. 6.Conduct formative and learning-oriented research, producing timely findings and highlighting their practical implications to inform ongoing implementation and support roll-out to sites in later waves of the programme. 7.Understand how MHSTs adapted their services and ways of working in response to the COVID-19 pandemic, and explore experiences of and learning from these changes, as well as their legacy. Methods We completed a mixed-methods evaluation combining quantitative and qualitative data collection across all 25 sites with in-depth qualitative insights from five purposively selected Trailblazers. The study comprised three work packages: •Work package 1: establishing the baseline and understanding the development and early impact of the Trailblazers. Participating education settings and key individuals who had a central role in the design and implementation of the MHSTs in their area were surveyed twice: December 2020 to May 2021 and October–November 2021. We received responses from 299 (30%, first survey) and 159 (17%, second survey) education settings; and from 76 (30%, first survey) and 65 (27%, second survey) key informants. We also interviewed the programme’s national leads (n establishing the baseline and understanding the development and en establishing the bn establishing the baseline and understanding the development and early impact of the Trailblazers. d documentation, and the development of demographic and mental health service profiles for the 25 sites, using publicly available data. •Work package 2: more detailed research with a range of stakeholders in five purposively selected Trailblazer sites, including focus groups with children and young people. A total of 71 interviews were completed with local stakeholders including MHST lead organisations and staff, school and college staff, individuals in Trailblazer governance and management roles, and wider partners including specialist NHS mental health services, voluntary organisations and local authorities. Five online focus groups were held with a total of 32 children and young people who attended schools where MHSTs were operating. •Work package 3: scoping and developing options for a longer-term assessment of the programme’s outcomes and impacts. This work was highly responsive and included reviewing the design and methods of recent evaluations of initiatives and pilots similar to the Trailblazers; ongoing advice and discussions with, and commentary on preparatory work undertaken by, the national programme team; a draft theory of change; and a full proposal for an initial impact evaluation. The Institute for Mental Health Youth Advisory Group at the University of Birmingham acted as an expert reference group for this research, and were involved throughout: from design through to preparation of this report. A key part of their role was co-producing the focus group research with children and young people, including co-designing the recruitment materials and topic guides, co-facilitating the focus groups and contributing to the analysis and presentation of the findings (see Chapter 9). Results Implementation and governance The Trailblazers had achieved a great deal in a relatively short space of time. While the local set-up process had been extensive, complex and rushed, some 12 months after the first cohort of EMHPs started their training all 58 MHSTs were operational in some form. The involvement of young people, parents and carers in the design and delivery of MHSTs was variable and often low, despite it being an aspiration that they be involved throughout the programme. There was a view that local governance and leadership was not yet truly shared across health, education and other key stakeholder groups and that the way in which the programme had been set up was dominated by the NHS as funder and by local mental health services. The pandemic created significant challenges for implementation, including delays to whole school activities; however, MHSTs adapted their offer and ways of working to ensure the continuation of support for young people and to education settings. These adaptations included the use of remote support. Stakeholders suggested that a hybrid model of in-person and remote delivery will be used going forward. The pandemic also had a considerable impact on the mental health and well-being of children and young people, and staff in education settings, as well as on access to specialist services. Children and young people described how home schooling had left them feeling disconnected, demotivated and sometimes without adequate support, as well as the difficulties transitioning back into school or college. Service models, delivery and gaps in support MHSTs were delivering a range of activities within the three core functions, with teams spending proportionally more time providing direct support than on their other two functions. Some teams were clinically oriented, while others took a more holistic/education-focused approach. The approach taken appeared to be most strongly influenced by the type of organisation(s) leading the programme (e.g. NHS vs. voluntary sector), and existing local infrastructure, relationships and skill sets. Teams also varied in the number of education settings they were working with, their staffing composition, and how whole school activities were being delivered (with this being led, in some areas, by specialist local partners or specialist roles within MHSTs). MHSTs had implemented strategies to reach and engage diverse groups and different mental health needs. However, stakeholders noted that some groups were underserved by MHSTs including children and young people with special educational needs or neurodiversity, those from ethnic minority backgrounds and some religious backgrounds, and children with challenging family or social circumstances (e.g. financial hardship, domestic abuse, or living in care). These issues concerning MHSTs’ reach and effectiveness were attributed to several factors, including gaps in the initial training programme and the limitations of the type of interventions that EMHPs had been trained to deliver (mainly time-limited, low-intensity cognitive behaviour therapy), which were felt to be poorly suited to some groups of children and young people and some mental health problems. Education settings were generally satisfied with the MHST service, and MHST staff spoke positively about working with education settings. However, a mismatch between education settings’ expectations or perceived support needs and what MHSTs could offer was sometimes reported, which hampered relationship building. Defining what was within the scope of ‘mild to moderate’ mental health was challenging, and practising within this scope was harder still. Some sites held a firm boundary around ‘mild to moderate’ mental health, whereas others provided support to children and young people with more serious and complex needs. There was a lack of clarity from programme regional and national leads about whether MHSTs should remain within their intended scope or offer flexibility to support children beyond this. Although MHSTs could refer young people with more complex needs to specialist services, there were long waiting times and restricted capacity in existing mental health services. Concern was expressed about children and young people falling through the gap between MHSTs’ ‘mild to moderate’ remit and the criteria for specialist support. Workforce and retention The EMHP role and training programme had been popular, but retaining EMHPs once in post was one of the biggest challenges reported by Trailblazers. Interviewees identified various reasons for poor retention including the role being seen as a stepping stone to other careers, lack of opportunities for career development and progression, frustration at the parameters of the role or limitations of the CBT approach and high workloads. Challenges recruiting senior team members were also common. There had been initial concerns about senior staff being recruited from other local mental health services, given the potential for this to create staffing shortages elsewhere in the local system, but many had come to the view that the movement of staff between services was positive inasmuch as it had helped build understanding and relationships. The degree of integration between MHSTs and specialist NHS services varied between areas, with some teams reporting a tension between working closely with other services and establishing a clear and distinctive identity within the diverse landscape of mental health providers in their area. Engagement and experiences of education settings, and children and young people Engagement of schools and colleges was felt to be critical to the success of the programme, as was the senior mental health lead role. Some education settings needed more help to prepare for the programme and make the most of the support on offer from their MHST, and there was disappointment about the delayed roll-out of the senior mental health leads training. Many education settings reported that constraints of time and competing commitments meant that mental health leads could not always engage with their MHST as much as they would have liked and this was a barrier to implementation and success. Children and young people were not always aware that there was an MHST in their education setting or what it did. Those who had had direct contact with the team (either receiving one-to-one support or through involvement in group or whole school activities) had a better understanding of MHSTs; their experiences of this contact had been universally positive and they were able to articulate more clearly how the school cared for their emotional well-being. Children and young people gave several examples of ways in which their education setting was promoting and supporting well-being for all pupils, and these were acknowledged and valued. Programme progress and outcomes Education settings reported positive early effects from participating in the programme, including staff feeling more confident talking to children and young people about mental health issues, being able to access advice about mental health issues more easily, and having quicker access to support for children and young people with some mental health problems. Improvements in children and young people’s understanding of mental health and well-being were also widely reported, as were strengthened relationships between education settings, mental health services and other local partners. Many education settings had invested in mental health support since joining the programme, although it was unclear whether this was a direct impact of the programme or due to other factors (e.g. a response to the COVID-19 pandemic). Various enabling factors critical to programme implementation and success were identified, including a supportive local context, multi-agency working to ensure that key organisations and sectors could influence the design and delivery of the programme, clear governance structures, sharing learning and co-production with children, young people and their families. Implications of the study findings for longer-term evaluation Key implications include: •There is considerable value in the longer-term evaluation focusing on understanding for which groups of children and young people, and which mental health problems, the standard MHST intervention is less suitable or beneficial. •Consideration must be given to which outcomes to measure, in consultation with children, young people, parents and carers. Some of the outcomes expected at the start of the programme may no longer be realistic, especially those relating to service use, given the impact of COVID-19. •Careful work will be required to define the programme’s ‘ecological’ impacts, and when these might be expected to occur since whole school effects are likely to be more diffuse and take longer to become visible. Limitations The study focused only on the first 25 Trailblazer sites in the programme. These sites were chosen for characteristics thought likely to drive rapid progress and learning and therefore the findings from this evaluation may not be reflective of experiences across the programme as a whole. Survey response rates were generally low, and some groups were less well represented in interview samples, including staff from educational settings and specialist NHS mental health services. The study did not include research to explore children and young people’s experiences of receiving mental health support from an MHST. Conclusions There have been substantial and unprecedented changes in the wider context since the programme started. The COVID-19 pandemic has further increased inequalities in mental health and access to support, and gaps between services appear to be widening. Critical decisions will need to be taken about what, if any, role MHSTs should have in providing support to children and young people beyond the ‘mild to moderate’ remit that the programme was designed to address. There is also the question of how the programme can continue to retain a dual focus on mental health promotion (e.g. through the development of whole school approaches) and early intervention, and what additional support or resources might help educational partners and settings maximise the opportunities offered by the programme. Alongside strategies for workforce creation and training, more work is needed to ensure that trained staff are retained and can develop in their roles. Funding The Birmingham, RAND and Cambridge Evaluation Rapid Evaluation Centre is funded by the NIHR Health Services and Delivery Research programme (HSDR 16/138/31). The Rapid Evaluation Centre and Policy Innovation and Evaluation Research Unit is funded by the NIHR Policy Research Programme (PR-PRU-1217-20602).
Keywords
