The Social and Emotional Education and Development intervention to address wellbeing in primary school age children: the SEED cluster RCT

Abstract

Background Stronger social and emotional well-being during primary school is positively associated with the health and educational outcomes of young people. However, there is little evidence on which programmes are the most effective for improving social and emotional well-being. Objective The objective was to rigorously evaluate the Social and Emotional Education and Development (SEED) intervention process for improving pupils’ social and emotional well-being. Design This was a stratified cluster randomised controlled trial with embedded process and economic evaluations. Thirty-eight primary schools were randomly assigned to the SEED intervention or to the control group. Hierarchical regression analysis allowing for clustering at school learning community level was conducted in R (statistical package). Setting The SEED intervention is a whole-school intervention; it involved all school staff and two cohorts of pupils, one starting at 4 or 5 years of age and the second starting at 8 or 9 years of age, across all 38 schools. Participants A total of 2639 pupils in Scotland. Intervention The SEED intervention used an iterative process that involved three components to facilitate selection and implementation of school-based actions: (1) questionnaire completion, (2) benchmarked feedback to all staff and (3) reflective discussions (all staff and an educational psychologist). Main outcome measure The primary outcome was pupils’ Strengths and Difficulties Questionnaire-Total Difficulties Score when pupils were 4 years older than at baseline. Results The primary outcome, pupils’ Strengths and Difficulties Questionnaire-Total Difficulties Score at follow-up 3, showed improvements for intervention arm pupils, compared with those in the control arm [relative risk −1.30 (95% confidence interval −1.87 to −0.73), standardised effect size −0.27 (95% confidence interval −0.39 to −0.15)]. There was no evidence of intervention effects according to deprivation: the results were significant for both affluent and deprived pupils. Subgroup analysis showed that all effect sizes were larger for the older cohort, particularly boys [relative risk −2.36 (95% confidence interval −3.62 to −1.11), standardised effect size −0.42 (95% confidence interval −0.64 to −0.20)]. Although there was no statistically significant difference in incremental cost and quality-adjusted life-years, the probability that the intervention is cost-effective at a willingness-to-pay threshold of £20,000 per quality-adjusted life-year was high, at 88%. Particularly valued mechanisms of the SEED intervention were its provision of time to reflect on and discuss social and emotional well-being and its contribution to a culture of evaluating practice. Limitations It was a challenge to retain schools over five waves of data collection. Conclusions This trial demonstrated that the SEED intervention is an acceptable, cost-effective way to modestly improve pupil well-being and improve school climate, particularly for older boys and those with greater levels of psychological difficulties. It was beneficial during the transition from primary to secondary school, but this diminished after 6 years. The SEED intervention can be implemented alongside existing systems for addressing pupil well-being and can be complementary to other interventions. Future work Assess whether or not the SEED intervention has a beneficial impact on academic attainment, is transferable to other countries and other organisational settings, would be strengthened by adding core training elements to the intervention process and is transferable to secondary schools. Understand the gender differences illustrated by the outcomes of this trial. Conduct further statistical research on how to handle missing data in longitudinal studies of complex social interventions. Trial registration This trial is registered as ISRCTN51707384. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 10/3006/13) and is published in full in Public Health Research; Vol. 12, No. 6. See the NIHR Funding and Awards website for further award information. Plain language summary We studied the Social and Emotional Education and Development (SEED) primary school intervention to see if it could improve the social and emotional well-being of pupils in Scotland. The SEED intervention is a process with several elements. We collected information from school pupils, staff and parents, and assessed if the schools involved were happy, safe and caring environments. We sought to highlight any strengths or weaknesses in how each school approaches social and emotional well-being. The SEED intervention also measures the social and emotional well-being of pupils. This includes pupils’ strengths and difficulties, confidence, understanding of emotions and quality of relationships. We gave the information back to each school to help them decide what they can do to improve the social and emotional well-being of their pupils. We gave schools a guide to available resources, reviewed according to how well they are known to work elsewhere. The same social and emotional well-being measurements were repeated every 1 or 2 years, to see if any improvements had been made, and to guide any further adaptions of activities. The study ran in 38 schools over 7 years; half of the schools were randomly selected to receive the SEED intervention and half carried on as normal. Two age groups of pupils were recruited; the younger group was aged 4 or 5 years and the older group was aged 8 or 9 years at the start of the study. We found that the SEED intervention did slightly improve social and emotional well-being. Improvements were greater for older pupils, in particular for boys, and lasted beyond their transition from primary to secondary school. We also found that it was cost-effective for schools to run the SEED intervention. Schools valued the structure and shared ownership associated with the process. We concluded that the SEED intervention is an acceptable way to modestly improve pupil well-being and school ethos. Scientific summary Background Improved social and emotional well-being (SEW) during primary school years has been shown to have an impact on health and academic performance and protect against risk behaviours in later years. However, there is little evidence on which school-based programmes are most effective, particularly in the UK. This trial rigorously evaluated one such innovative programme in Scotland. The Social and Emotional Education and Development (SEED) intervention was designed to promote SEW in primary schools in Scotland. Rather than adopting a one-size-fits-all approach whereby SEW packages are delivered in schools to all pupils regardless of need, the SEED intervention was designed to draw on principles of co-production to tailor activities in response to school need. It has three components: (1) an assessment of school needs through staff, pupil and parent questionnaires; (2) feedback of needs assessment data to all school staff and reflective discussion (RD), facilitated by the schools’ educational psychologists (EPs), to select and co-produce school-appropriate, evidence-based actions and initiatives at both class and whole-school levels; and (3) implementation and maintenance of initiatives. Objectives The overarching aim of this study was to rigorously evaluate the impact of the SEED intervention on improving pupils’ SEW via a stratified cluster randomised controlled trial (RCT). The main research questions were addressed by the complementary outcome, process evaluations and economic evaluations. The pupil-related research questions were as follows: Does the SEED intervention improve pupils’ SEW? If so, is the impact different for specific subgroups of pupils (e.g. gender, deprivation)? Is the SEED intervention more effective if started with younger children [SEED trial younger cohort (YC) vs. older cohort (OC)]? What is the duration of the SEED intervention effect? Does the SEED intervention improve the social and emotional experience of transition from primary to secondary school? What is the impact on health behaviours of the SEED intervention during early secondary school years? What are pupils’ experiences of the SEED intervention? The teacher-related research questions were as follows: Are there changes in teachers’ knowledge, attitudes and behaviour relating to developing pupils’ SEW? Were teachers involved, and, if so, how were they involved, in selecting initiatives to respond to the pupils’ needs assessment? What contextual factors facilitate or inhibit the delivery of the SEED intervention? What contextual factors support or hinder the ability of the SEED intervention to improve pupils’ SEW? Which teachers engage best with the SEED intervention? What are teachers’ experiences of the SEED intervention? The parent-related research questions were as follows: Do parents report a difference in their child(ren)’s emotional and social development? If applicable, what are parents’ experiences of the SEED intervention? The economic research question was as follows: is SEED cost-effective? Methods A stratified cluster RCT was undertaken across 38 schools in the central Scotland area between 2013 and 2019. We invited state-funded denominational and non-denominational schools from three Scottish local authorities; we did not include independently funded schools. The intervention was delivered in primary schools, but the evaluation took place in primary and secondary schools. Baseline questionnaire data were collected from two cohorts of pupils in primary school, one in year 1 (aged 4–6 years) and the other in year 5 (aged 8–10 years), their parents and school staff. After a 1-year gap, to enable commencement of action plans, three waves of follow-up data were collected annually, then a further final follow-up wave was carried out 2 years after that with the OC of pupils only. Main trial outcomes The primary outcome was the Strengths and Difficulties Questionnaire-Total Difficulties Score (SDQ-TDS) at time 3 (T3) (follow-up 3), when the younger pupils were aged 8–10 years and the older pupils were aged 12–14 years. Secondary outcomes included all five subscales of the Strengths and Difficulties Questionnaire (SDQ); additional measures included pupil and staff well-being, school ethos and relationships. Hierarchical regression analysis allowing for clustering at school learning community level was conducted in the statistical package R (The R Foundation for Statistical Computing, Vienna, Austria). Missing data were handled using repeated measures. Process evaluation The process evaluation design was guided by the UK Medical Research Council framework for the evaluation of complex interventions. Mixed methods included semistructured interviews with head teachers/depute head teachers and EPs, ethnographic notes from RD sessions and focus groups with pupils. An initial analysis was carried out on data collected from case study schools prior to knowing the main trial outcomes using an agreed coding framework. Further thematic analysis was conducted using NVivo (QSR International, Warrington, UK) to interpret trial outcomes and understand mechanisms of change. Economic evaluation The economic evaluation was conducted from a public sector perspective, including the NHS, Personal Social Services and local government. A validated child-specific preference-based measure, the Child Health Utility-9 Dimensions, was used to obtain utility values, which, together with costs, were used to conduct a cost–utility analysis. In addition, a within-trial cost-effectiveness analysis was conducted using the SDQ-TDS, the primary outcome measure for the trial. Data on the costs include costs associated with the provision of the intervention, including staff time, travel and consumables at each stage of the intervention, as well as the resource use cost collected at time 1 (T1) (follow-up 1) and T3 in the public sector, such as general practitioner visits and police visits. Resource use data were collected via the self-completed questionnaires, and the unit costs were obtained from standard UK sources. All analyses were undertaken according to the principle of intention-to-treat and in Stata®/SE 14.0 (StataCorp LP, College Station, TX, USA). Results (research findings) We recruited 38 primary schools with > 2600 pupils. The primary outcome, pupils’ SDQ-TDSs at T3, showed statistically significant improvements for pupils in the intervention arm, compared with those in the control arm [relative risk −1.30, 95% confidence interval (CI) −1.87 to −0.73]. There was no evidence of intervention effects according to deprivation: the results were significant for both affluent and deprived pupils. A subgroup analysis showed that all effect sizes were larger for the OC, particularly older boys (relative risk −2.36, 95% CI −3.62 to −1.11). All five SDQ subscales also showed beneficial and statistically significant results. A secondary analysis showed that the intervention had a range of positive effects on social and emotional skills, and school ethos and relationships. Only boys showed significant results for conduct problems and prosocial behaviour; all the other subscales showed effects for both genders. The intervention effect appeared to be diminished at the longer-term, final follow-up, 6 years post baseline. The economic evaluation concluded that the SEED intervention was likely to be cost-effective, but more so for the OC than the YC. Among the YC, the intervention was associated with a £166 increase in the cost and a 0.19 decrease in the SDQ-TDS, leading to a result of £856 per decrease in SDQ-TDS. Costs were lower in the OC because of the smaller incremental cost and much larger treatment effect, with an incremental cost-effectiveness ratio of £1.22 per 1-unit decrease in the SDQ-TDS. The probability of the SEED intervention being cost-effective at £20,000 per quality-adjusted life-year was > 80%. The process evaluation showed that there was an appreciation for the timing of the SEED intervention in terms of its alignment with the Scottish curriculum. The intervention was perceived as complementary to, rather than competing against, other established initiatives and priorities. Although there was variability in fidelity to the SEED intervention process, the data collection and the process of feeding back data at whole-school staff sessions were the most consistently implemented aspects. Particularly valued mechanisms of the SEED intervention were its provision of time to reflect on and discuss SEW and its contribution to a culture of evaluating practice. Limitations It was a challenge to retain schools over five waves of data collection. At two waves of data collection (T2 and T4), we lost three schools; however, we did have good school participation at the other three waves of data collection. We caution readers that there is more uncertainty about our results for these two waves. Low parental participation in our survey meant that parent data were unlikely to be a fair basis for influencing school-level decision-making, and this was communicated to schools. Conclusions The SEED cluster RCT provides robust evidence about the effects of the SEED intervention, a school-based intervention to promote SEWamong children. We have demonstrated that the SEED intervention is an acceptable, cost-effective way to modestly improve pupil well-being and improve school climate, particularly for older pupils and, among the older pupils, particularly for boys and those with greater levels of psychological difficulties of both genders. There is no evidence of the intervention being more beneficial for advantaged pupils, so it should not widen the inequalities gap. It was beneficial during the transition from primary to secondary school, but longer-term analysis suggests that the effect is diminished after 6 years. The SEED intervention can be implemented alongside existing systems for addressing pupil well-being and can be complementary to other intervention initiatives. Future work Our recommendations for future implications in relation to action, in numbered order, are to: conduct data linkage to SEED pupils’ external examination results (aggregated linkage rather than individual-pupil level), to assess whether or not the SEED intervention had any beneficial impact on academic attainment explore the transferability of the SEED intervention to other countries within the UK and beyond investigate the impact of adding core training elements (e.g. teacher training on young people’s mental health and well-being) to the intervention process extend the SEED intervention-style process to secondary schools understand the gender differences illustrated by the SEED outcomes develop and evaluate the SEED intervention concept in different settings across the life course further statistical research on how to handle missing data in longitudinal studies of complex social interventions. Trial registration This trial is registered as ISRCTN51707384. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 10/3006/13) and is published in full in Public Health Research; Vol. 12, No. 6. See the NIHR Funding and Awards website for further award information.