Public Health Research (Dec 2024)
Interventions to improve mental health and well-being in care-experienced children and young people aged less than 25: the CHIMES systematic review
Abstract
Background Children and young people with experience of being in care (e.g. foster care, kinship care, residential care or at home with a supervision requirement order) are at higher risk of adverse mental health and well-being outcomes compared to the general population. Despite a range of policy recommendations and interventions, it is not clear what approaches are effective in the United Kingdom, or how context factors give rise to facilitators and inhibitors of implementation and acceptability. Objectives The CHIMES review is a complex-systems-informed mixed-method systematic review that aimed to synthesise the international evidence base for interventions addressing the mental health and well-being of care-experienced children and young people (age ≤ 25 years) and to assess the potential transportability of this evidence base to the United Kingdom context. Data sources We searched 16 electronic bibliographic databases and 22 websites from 1990 to May 2022. We conducted citation tracking, screened relevant systematic reviews and contacted international experts. Method We used a convergent synthesis design. We first constructed an evidence map to confirm review scope before undertaking method-level syntheses for outcome evaluations, process evaluations and economic evaluations. These elements were integrated into a review-level synthesis to identify potential evidence-based interventions that may progress to further development, adaptation and evaluation in the United Kingdom. We conducted stakeholder consultations to prioritise intervention theories, types and outcomes. Results We identified 64 interventions from 124 study reports. Interventions were primarily evaluated in the United States and targeted young people’s competencies or carers’ parenting practices. Meta-analysis reported limited evidence that interventions effectively improved mental health in the shorter term (0–6 months): total social, emotional and behavioural problems (d = −0.15, 95% confidence interval −0.28 to −0.02); internalising problem behaviours (d = −0.35, 95% confidence interval −0.61 to −0.08); externalising problem behaviours (d = −0.30, 95% confidence interval −0.53 to −0.08); depression and anxiety (d = −0.26, 95% confidence interval −0.40 to −0.13) and social–emotional functioning difficulties (d = −0.18, 95% confidence interval −0.31 to −0.05), but these impacts were not observed in the longer term (> 6 months). Five key context factors potentially explain challenges to implementation and acceptability: lack of system resources; the time, cognitive and emotional burden of delivery or participation; interprofessional tensions; the devaluing of young people, meaning that they felt unable to express dissatisfaction with interventions; and the devaluating of carers’ expertise and needs. From the evidence, stakeholder consultation identified two priority interventions: (1) mentoring by individuals with knowledge and experience of care and (2) system and ethos change to create harmonisation between organisations and facilitate interprofessional relationships. Well-being and suicide-related behaviours are priority outcomes alongside mental health. Limitations The review was limited by a paucity of theory and economic evaluations, so it is unclear how interventions might function or their potential cost-effectiveness. Interventions were insufficiently described, making it challenging to map the evidence base. Outcome evaluations were poorly reported. Due to ongoing restrictions with COVID-19, stakeholder consultations were conducted later than intended with a smaller number of attendees. Conclusions The review identified some evidence for interventions impacting mental health in the short term. There is a lack of system-level interventions and approaches that target subjective well-being and suicide-related outcomes. Future intervention might prioritise mentoring and targeting system culture. Study registration This study is registered as PROSPERO CRD42020177478. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: NIHR129113) and is published in full in Public Health Research; Vol. 12, No. 14. See the NIHR Funding and Awards website for further award information. Plain language summary Children and young people in care may include those who live with foster carers, kinship carers or in residential care. Some care-experienced young people will have poorer mental health and well-being than those who have not been in care. In the United Kingdom, there have been policies and recommendations to improve mental health service provision, but it is not clear what works. We brought together all international research to understand what approaches work best and if they could be successfully used in the United Kingdom. We found 64 different types of mental health and well-being provision for children and young people up to the age of 25 years. Most approaches were delivered and evaluated in the United States. Generally, there was a lack of description of different provisions, but they often focused on improving young people’s social and emotional skills or changing the parenting style of foster carers. We found that, in the short term, these approaches could work to positively impact children and young people’s mental health, but in the longer term they did not seem to work. It is not clear if these approaches are more cost-effective than current services. We did not find much research looking at the impact on children and young people’s well-being, self-harm or suicide. We also considered some of the challenges to delivering services. Issues included a lack of time and finance, the burden on social care staff and carers, difficulty in relationships between organisations, services not meeting the needs of young people and carers struggling to deliver new approaches that do not fit with their skill set. After talking through our findings with stakeholders, we identified potential services that should be considered for delivery in the United Kingdom. These are mentoring for children and young people by individuals with experience of care and support to help carers, other professionals and organisations to work together. Scientific summary Background Care-experienced children and young people may be defined as individuals who have resided in kinship care, foster care, residential care or who remain at home but with statutory intervention that transfers legal parental rights to local authorities. They are reported to experience adverse mental health and well-being outcomes in comparison with the general population. Despite policy and guidance recommendations to improve the quality of support provided to care-experienced young people in the UK, the current evidence base for intervention in this context is limited. This is in contrast to a more comprehensive, if equivocal, evidence base internationally, particularly in the USA. There is a clear need for evidence syntheses that draw together evaluations reporting the effectiveness of different types of intervention approaches, while also exploring the contexts in which they are delivered and evaluated. Such work would help researchers and policymakers better understand the potential transportability of international evidence-based approaches beyond their immediate evaluation contexts, specifically to the UK. It is then important to establish the extent to which de novo intervention development, adaptation or revaluation is required for the UK setting. Objectives The Care-experienced cHildren and young people’s Interventions to improve Mental health and wEll-being outcomes Systematic review (CHIMES) review is a complex-systems informed, multimethod systematic review that aimed to synthesise extant international evidence on interventions addressing the mental health and well-being of care-experienced children and young people. This research aim was addressed through the following research questions (RQs): What are the types, theories and outcomes tested in mental health and well-being interventions for care-experienced children and young people? What are the effects (including inequities and harms) and economic effects of interventions? How do contextual characteristics shape implementation factors and what are key enablers and inhibitors of implementation? What is the acceptability of interventions to target populations? Can and how might intervention types, theories, components and outcomes be related in an overarching system-based programme theory? Drawing on the findings from RQ1 to RQ5, what do stakeholders think is the most feasible and acceptable intervention in the UK that could progress to further outcome or implementation evaluation? Methods We conducted a mixed-method systematic review, adopting a convergent synthesis design. This approach entailed method-specific syntheses conducted in a complementary manner, which were subsequently integrated into a further review-level synthesis. Data sources We searched 16 electronic bibliographic databases and 22 websites from 1990 to May 2022. A total of 32 subject experts and 17 third-sector organisations were contacted to identify additional grey literature, unpublished research or ongoing studies. We screened relevant systematic reviews identified at the protocol development stage and through the searches of electronic bibliographic databases. We conducted backward and forward citation tracking of included study reports. Data extraction We coded all eligible study reports as part of the review mapping, with intervention descriptions being coded using the Template for Intervention Description and Replication (TIDieR) checklist. Process evaluations were extracted according to context, implementation and acceptability. A subset of conceptually and/or empirically richer process evaluations were extracted according to the context and implementation of complex interventions framework, which classifies pertinent context domains. For outcome evaluations, randomised controlled trials (RCTs) and non-randomised studies were extracted according to study arms, analysis and outcomes, with study design-specific features also being coded. Equity harms were extracted from study reports that included moderator analysis or interaction effects. Harms were initially categorised according to the PROGRESS-Plus for equity harms. Economic evaluations coded according to the Drummond checklist. Quality appraisal We appraised programme theory study reports using a tailored appraisal tool developed for a previous systematic review with theory synthesis. Qualitative data within rich process evaluations were appraised using a tool developed in a previous systematic review, assessing reliability and trustworthiness. Outcome evaluations that were conducted using a RCT study design were appraised using the Cochrane risk of bias tool for randomised trials (RoB 2). Outcome evaluations that were conducted using a non-randomised study design were appraised using the Cochrane Risk Of Bias In Non-randomized Studies – of Interventions. For the assessment of certainty, we used Grading of Recommendations Assessment, Development and Evaluation (GRADE) and GRADE-Confidence in the Evidence from Reviews of Qualitative Research tools. Data synthesis Following the identification of eligible study reports, we constructed an evidence map to confirm the review scope and identify reports to be included in method-specific syntheses. Rich process evaluations were synthesised with framework synthesis. Thin-process evaluations, usually integrated with outcome evaluations, were descriptively summarised. For eligible RCT studies, we conducted meta-analyses for outcome domains relating to mental, behavioural or neurodevelopmental disorders as specified by the International Classification of Diseases, 11th Edition. There was not an adequate number of studies to conduct meta-analyses for the outcome domain of subjective well-being or suicide-related outcomes. We constructed narrative overviews for equity harms, with harvest plots for interventions targeting mental health, behavioural and neurodevelopmental disorders, as there was a sufficient number of study reports. Due to a lack of eligible economic evaluations, we narratively summarised one partial evaluation. We integrated the method-level syntheses into a review-level synthesis at two key points. First, we integrated the synthesis of thin and rich process evaluations (RQ3–4) with outcome data to explain intervention effectiveness and variations in effects (RQ2). Second, we constructed two integrative matrices. The first of these 2 × 2 matrices mapped interventions by stakeholder preferences (both in process evaluations and consultations) in regard to intervention theories and types. This was intended to identify whether the designs of interventions are relevant and responsive to needs within the UK context. The second mapped intervention outcomes by stakeholder priority outcomes to assess whether interventions are targeting the right domains. Stakeholder consultations At commencement of the review, we conducted stakeholder consultations with advisory groups of care-experienced young people and a foster carer manager advisory group to refine and confirm the review scope. Following completion of the method-level syntheses, we undertook seven stakeholder consultations with: two care-experienced young people’s advisory groups, one foster carer group, three health and social care practitioner groups and one government group. These consultations reflected on the evidence base and the potential transportability to the UK context. They considered whether identified intervention theories and types could be effective, feasible and acceptable in the UK, or if de novo developmental or adaptation would be required (RQ6). Results What are the types, theories and outcomes tested in mental health and well-being interventions for care-experienced children and young people? In total, 15,068 unique study reports were identified. Following screening, 64 interventions with 124 associated study reports were eligible for inclusion in the review. Study reports were published between 1994 and 2022, with the majority conducted solely in the USA (n = 77) or the USA and UK (n = 1). There were 24 study reports describing interventions’ programme theory, 50 process evaluations reporting context, implementation and acceptability, 86 outcome evaluations and 1 partial economic evaluation. We classified interventions according to the socioecological domains in which they operated, working on the assumption that they may interact with contextual characteristics differently depending on the part of the system they targeted. Of the interventions, 9 targeted the intrapersonal level, 15 targeted both the intrapersonal and interpersonal domain, 1 targeted the intrapersonal, organisational and community domains, 26 targeted the interpersonal domain, targeted the interpersonal and organisational domain, 5 targeted the interpersonal, organisational and community domain, 1 targeted the organisational domain, 4 targeted the community domain and 1 targeted the policy domain. The 13 interventions reporting a programme theory were mainly relational and focused on attachment theory, positive youth development and social learning theory. This reflected the predominance of interpersonal interventions in the review. There was also system change theories linked to interventions operating at the higher socioecological domains, although these generally focused on restructuring the system to support interpersonal approaches. Interventions primarily targeted mental health, behavioural and neurodevelopmental disorders. The most frequently assessed outcome measurements were total social, emotional and behavioural problems (n = 48); social–emotional functioning difficulties (n = 17); externalising problem behaviours (n = 26) and internalising problem behaviours (n = 22). Only 11 interventions targeted subjective well-being and 4 targeted suicide-related behaviours. What are the effects (including inequities and harms) and economic effects of interventions? We synthesised evidence from 44 RCT evaluations of 35 interventions. Meta-analyses showed that interventions reporting outcomes for up to 6 months post baseline demonstrated some effectiveness for reducing children and young people’s: total social, emotional and behavioural problems [d = −0.15, 95% confidence interval (CI) −0.28 to −0.02]; internalising problem behaviours (d = −0.35, 95% CI −0.61 to −0.08); externalising problem behaviours (d = −0.30, 95% CI −0.53 to −0.08); depression and anxiety (d = −0.26, 95% CI −0.40 to −0.13) and social–emotional functioning difficulties (d = −0.18, 95% CI −0.31 to −0.05). Assessment of evidence using GRADE showed low or very low certainty across outcome domains, primarily relating to concerns arising from risk of bias and imprecision across evaluation reports. For outcome domains where there were a sufficient number of effect sizes to evaluate longer-term (> 6 months) outcomes (total social, emotional and behavioural problems; internalising problem behaviours; externalising problem behaviours and social–emotional functioning difficulties), we found no evidence that interventions demonstrated effectiveness. Evidence of equity harms indicated limited differential outcomes according to population groups. However, there was some tentative indication that interventions targeting mental health, behavioural and neurodevelopmental disorders were more beneficial for those with less exposure to maltreatment and those with more severe baseline mental health problems. The review only identified one partial evaluation of an included intervention assessing intervention costs in the UK relative to the USA. How do contextual characteristics shape implementation factors, and what are key enablers and inhibitors of implementation? What is the acceptability of interventions to target populations? We categorised process evaluations according to conceptually and/or empirically thin (n = 27) or rich (n = 23). Generally, thin-process evaluations indicated that interventions had high fidelity and acceptability, although there were reported issues with recruitment and retention. From rich-process evaluations, we generated five key context themes that might serve as facilitators or inhibitors to implementation and acceptability: (1) lack of system resources; (2) the time, cognitive and emotional burden of delivery and participation; (3) tensions in interprofessional relationships; (4) the systemic devaluing of care-experienced young people where their needs and preferences are not prioritised and (5) the discounting of carers’ expertise, knowledge and other potentially conflicting commitments, which can mean that interventions do not fit with the wider context of their lives. There was no clear difference between interventions that reported high levels of implementation and/or acceptability and effectiveness. Can and how might intervention types, theories, components and outcomes be related in an overarching system-based programme theory? From our mapping and synthesis of theory, outcome and process evaluations, we identified three clusters of interventions that might have potential to progress to further testing in the UK. Within these clusters of intervention types, there were specific programme theories or components that demonstrated some evidence of effectiveness. Two of these approaches primarily operate at the interpersonal level: (1) mentoring interventions delivered by care-experienced peers or significant adults with knowledge or experience of care and (2) parenting interventions, largely targeted at foster and kinship carers, that provide training and support in parenting skills, knowledge and practices. The third type targets the organisational and community domains, and comprises system-change interventions facilitating interorganisational relationships and collaboration, largely through the harmonisation of ethos. Currently, these types of interventions primarily target mental health, behavioural and neurodevelopmental disorders, although there are examples of each intervention type addressing subjective well-being and self-harm. From the evidence base, these approaches have not been combined into an overarching intervention model, although they are not theoretically discordant and might have the potential for integration. Drawing on the findings from research questions 1–5, what do stakeholders think is the most feasible and acceptable intervention in the United Kingdom that could progress to further outcome or implementation evaluation? Reflecting on the evidence synthesis, consultations with stakeholders refined key context factors, in addition to identifying priority intervention theories, types and outcomes that can inform further intervention development, adaptation and evaluation in the UK. Stakeholders confirmed the key context factors generated by the process evaluation synthesis as being relevant facilitators or inhibitors to intervention implementation and acceptability in the UK. Priority intervention types emphasised mentoring approaches, preferably by care-experienced peers, and system change approaches where harmonisation in ethos across professional groups and community organisations works to facilitate interagency working in decision-making and service co-ordination. These intervention types may be underpinned by theories that have an emphasis on positive relationships (e.g. attachment theory, positive youth development, and social learning theory) and progress understanding of the particular challenges and complexities experienced by young people in care (e.g. trauma-informed practice). Parenting interventions were not considered a priority where they were theoretically aligned with behavioural management. Priority outcomes for stakeholders were subjective well-being and suicide-related behaviours. Interventions may be adapted to also assess these outcomes where theoretically appropriate or de novo development may be required. Conclusions The available evidence base reporting on interventions targeting the mental health and well-being of care-experienced children and young people is mixed, and is limited for certain intervention theories, types and outcomes. The evidence base, primarily from the USA, focuses on intrapersonal and interpersonal approaches that develop the skills and knowledge of young people and their carers. Current interventions primarily target mental, behavioural and neurodevelopmental disorders. We identified mentoring and system ethos change interventions as being a priority in the UK context, provided additional developmental and adaptation work is undertaken to sensitise these types of approaches to local system needs. Study registration This study is registered as PROSPERO CRD42020177478. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: NIHR129113) and is published in full in Public Health Research; Vol. 12, No. 14. See the NIHR Funding and Awards website for further award information.
Keywords
