Public Health Research (Sep 2024)
An ‘alcohol health champions’ intervention to reduce alcohol harm in local communities: a mixed-methods evaluation of a natural experiment
Abstract
Background Globally alcohol consumption is a leading risk factor for premature death and disability and is associated with crime, social and economic consequences. Local communities may be able to play a role in addressing alcohol-related issues in their area. Objectives To evaluate the effectiveness and cost–benefit of an asset-based community development approach to reducing alcohol-related harm and understand the context and factors that enable or hinder its implementation. Design A mixed-methods evaluation. Area-level quasi-experimental trial analysed using four different evaluation methods (a stepped-wedge design where each area was a control until it entered the intervention, comparison to matched local/national controls and comparison to synthetic controls), alongside process and economic evaluations. Setting Ten local authorities in Greater Manchester, England. Participants The outcomes evaluation was analysed at an area level. Ninety-three lay persons representing nineareas completed questionnaires, with 12 follow-up interviews in five areas; 20 stakeholders representing ten areas were interviewed at baseline, with 17 follow-up interviews in eight areas and 26 members of the public from two areas attended focus groups. Interventions Professionals in a co-ordinator role recruited and supported lay volunteers who were trained to become alcohol health champions. The champion’s role was to provide informal, brief alcohol advice to the local population and take action to strengthen restrictions on alcohol availability. Main outcome measures Numbers of alcohol-related hospital admissions, accident and emergency attendances, ambulance call-outs, street-level crime and antisocial behaviour in the intervention areas (area size: 1600–5500 residents). Set-up and running costs were collected alongside process evaluation data exploring barriers and facilitators. Data sources Routinely collected quantitative data on outcome measures aggregated at the intervention area and matched control and synthetic control areas. Data from policy documents, licensing registers, meeting notes, invoices, time/cost diaries, training registers, questionnaires, interviews, reflective diaries and focus groups. Results The intervention rolled out in nine out of ten areas, seven of which ran for a full 12 months. Areas with better-established infrastructure at baseline were able to train more champions. In total, 123 alcohol health champions were trained (95 lay volunteers and 28 professionals): lay volunteers self-reported positive impact. Champions engaged in brief advice conversations more readily than taking action on alcohol availability. There were no consistent differences in the health and crime area-level indicators between intervention areas and controls, as confirmed by using three different analysis methods for evaluating natural experiments. The intervention was not found to be cost-beneficial. Limitations Although the sequential roll-out order of the intervention was randomised, the selection of the intervention areas was not. Self-reported impact may have been subject to social desirability bias due to the project’s high profile. Conclusions There was no measurable impact on health and crime outcomes. Possible explanations include too few volunteers trained, volunteers being unwilling to get involved in licensing decisions, or that the intervention has no direct impact on the selected outcomes. Future work Future similar interventions should use a coproduced community outcomes framework. Other natural experiment evaluations should use methodological triangulation to strengthen inferences about effectiveness. Trial registration This trial is registered as ISRCTN81942890. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 15/129/03) and is published in full in Public Health Research; Vol. 12, No. 9. See the NIHR Funding and Awards website for further award information. Plain language summary What was the question? Alcohol consumption puts an individual’s health and social relationships at risk of harm. The more a person drinks, the more harmful it is. The harmful effects can place a burden on emergency services and hospitals. We wanted to find out whether community members can make a difference by taking action to address alcohol harm in their local area. What did we do? Local councils in Greater Manchester developed a project called Communities in Charge of Alcohol, where volunteers in targeted local areas were trained to become ‘alcohol health champions’. Alcohol health champions gave alcohol-related brief advice to people to help them drink less. They had a say about when, where and how alcohol is sold by reporting issues to their local council. We compared numbers of alcohol-related hospital admissions, accident and emergency attendances, ambulance call-outs and reports of crime and antisocial behaviour between areas that had alcohol health champions with other similar areas in England that did not. We calculated how much it costs to run and whether Communities in Charge of Alcohol could save society money. What did we find? Not as many volunteers came forward to become an alcohol health champion as hoped for. Those who did give alcohol-related brief advice to people. They preferred not to report issues about alcohol sales to their local council, either because it was too complicated or because they did not want to be called a ‘grass’. We did not find levels of alcohol harm changed in the Communities in Charge of Alcohol areas. Because of this, we could not demonstrate that Communities in Charge of Alcohol could save society money. What does this mean? Getting involved in alcohol licensing decisions needs to be made easier for communities, with more anonymity, through the support of professionals. More work needs to be done to understand whether giving brief advice can reduce alcohol harm in whole communities. Scientific summary Background The availability of, and access to, alcohol is a key determinant of alcohol harm, and powers exist within the Licensing Act 2003 of England and Wales that enable the public to influence the local sale and supply of alcohol. At an individual level, implementing programmes to promote early risk identification and brief advice have the potential to reduce alcohol consumption and prevent alcohol-related harm, but there is a dearth of evidence to determine the feasibility and effectiveness of lay people having a role. The Greater Manchester (GM) Combined Authority (GMCA) and Greater Manchester Health and Social Care Partnership (GMHSCP) designed a new programme, ‘Communities in Charge of Alcohol’ (CICA) in 2017 to train lay volunteers to become alcohol health champions (AHCs), who would be able to (1) give alcohol-related brief advice to individuals and (2) help communities influence alcohol availability and strengthen restrictions in alcohol risk environments. Overall aims To evaluate the effectiveness and cost–benefit of implementing a locally delivered AHC intervention and understand the context and factors that enable or hinder the intervention. Objectives Relating to the outcome evaluation: to determine the effect on area-level key health performance indicators: alcohol-related hospital admissions (narrow measure), weekend accident and emergency (A&E) attendances (Friday 3 p.m.–Sunday 3 p.m.), weekday A&E attendances (Monday–Friday 9 a.m.–5 p.m. each day), weekend ambulance call-outs (Friday 3 p.m.–Sunday 3 p.m.), weekday ambulance call-outs (Monday–Friday 9 a.m.–5 p.m. each day) to determine the effect on key crime indicators (street-level crime data) at the weekend (Friday 3 p.m.–Sunday 3 p.m.) and weekday (Monday–Friday 9 a.m.–5 p.m. each day) to determine the effect on key antisocial behaviour (ASB) indicators (police recorded calls for service were classified as ASB according to the National Standard for Incident Recording). Relating to the process evaluation: to explore policy context and variation in licensing practice, including any impact of devolution in GM to explore barriers and facilitators at key stages of the implementation of the intervention: recruitment of AHCs to initial training and cascade training, delivery of initial training and cascade training, use of skills beyond the training in AHC activity, retention of AHCs to explore responses to AHC training, modelling of health behaviours, perceptions of community cohesion and development to determine numbers of brief interventions applied and community awareness events organised/participated in to examine and quantify the amount and success of community involvement in licensing issues to determine whether there was a change in composite measures of alcohol availability. Relating to the economic evaluation: to identify set-up and running costs using a standardised costing exercise to resolve costs by sector (health, ambulance and police) before, during and after set-up to quantify benefits due to reduced hospital admissions, ambulance call-outs, A&E use, crime and ASB. Methods Communities in Charge of Alcohol was a complex intervention already planned by the GMCA in 10 small local authority areas (~1600–5500 residents each) and was outside direct researcher control. Intervention areas were chosen by each local authority public health team as having high levels of alcohol harm in comparison to the rest of the local authority. All intervention areas were at the bottom of the deprivation index, in deprivation decile 1 or 2. Using a stepped-wedge design, researchers were able to randomise the order of roll-out of intervention areas to bring a quasi-experimental approach to the evaluation. Two additional evaluation designs were used in order to triangulate findings, namely comparison to matched controls and comparison to synthetic controls. Ethical approval was received from the University of Salford Research Ethics Committee on 17 May 2017 (reference number: HSR1617–135) and obtained from the University of Bristol on 16 May 2019 (reference number: 82762). Outcome evaluation data sources Routinely collected data at the lower-layer super output area (LSOA) level for intervention areas was compared with control areas. Data sources were as follows: alcohol-related hospital admissions data – Public Health England (now Office for Health Improvement and Disparities) accident and emergency department attendance data – NHS Digital ambulance call-out data – North West Ambulance Service reported crime and ASB data – GM Police. Analysis spanned a 10-year period (7 years pre intervention and a maximum of 3 years post implementation), ending in January 2020. Data were analysed using log-rate growth models and time series analyses to quantify the effect of the CICA programme on key performance indicators. Process evaluation data sources Pre-implementation phase document review of local Statements of Licensing Policy (n = 9), alcohol availability composite score of on- and off-licensed premises (n = 9 areas), roll-out preparatory meeting notes. Train-the-Trainer attendance registers by area (n = 5, representing 9 areas, attended by 48 lay participants and 25 professional participants). Cascade training attendance registers by area (n = 11, representing 7 areas, attended by 47 lay participants and 3 professional participants). Pre- and post-training questionnaires (n = 93 lay people, 98% response rate). Baseline interviews with stakeholders (n = 20) (initial implementation phase); interviews with lay volunteer AHCs (n = 5, representing 3 areas) within 3–6 months of initial training (implementation phase). Follow-up interviews (12 months) with stakeholders (n = 11, representing eight areas), licensing officers (n = 6, representing six areas) and lay volunteer AHCs (n = 7, representing four areas); follow-up questionnaires with AHCs (n = 11); focus groups with local residents (n = 3 groups, representing two areas, with a total of 26 participants). Economic evaluation data sources Time and transportation costs incurred during project meetings aggregated from meeting minutes and stakeholder time/cost diaries. Invoices for the design, development and delivery of training packages and accreditation fees. Invoices for the qualification and assessment fees, time and miscellaneous (transportation, room hire, catering, printing costs). AHC time and costs aggregated from volunteer diaries and stakeholder time/cost diaries. Invoices for engagement materials (AUDIT-C scratchcards). Greater Manchester Combined Authority research team’s (formerly New Economy) cost–benefit analysis (CBA) tool was populated using aggregated outcome evaluation data to resolve costs for each key performance indicator before, during and after CICA setup. Results Nine out of 10 local authorities rolled out the CICA intervention on their given start date: seven areas completed a full year, carrying out cascade training in the first 12 months, two areas withdrew at 6 and 9 months and one area withdrew in the pre-implementation phase. There were 118 [interquartile range (IQR): 60–205] alcohol-related hospital admissions per month per pre-intervention LSOA, rising to 134 (IQR: 67–203) post intervention. Some outcomes had very small counts at the LSOA level; notably, there were only 2 (IQR: 1–3) weekend crimes per month pre-intervention and 4 (IQR: 2–6) post intervention. The primary, stepped-wedge analyses provide weak evidence of an average increase in alcohol-related hospital admissions following the implementation of CICA of about 13%, corresponding to about 20 admissions weekly, although the confidence interval was wide, −1.98 to 31.39, and not statistically significant (p~0.09), while the analysis that made use of local controls suggested a bigger effect, at 16.4% (7.33–26.16, p < 0.001). However, the other methods of analysis (national controls and counterfactuals) indicated much smaller effect sizes and confidence intervals, including unity: 3.42% (−4.56 to 12.07; p = 0.41) and 7.14% (−9.76 to 24.04; p = 0.41), respectively. The other quantitative outcomes (A&E admissions, or ambulance call-outs to the area, nor to reported crimes or reported ASB incidents) showed some were individual statistically significant effects (with some methods and some indicators), but these were more often not in favour of the intervention. Triangulation of three methods of analysis did not indicate any consistent differences between control and intervention areas. The pre-implementation phase was defined as the period leading up to the roll-out of the first Train-the-Trainer event. Several facilitating contextual factors were identified a priori. The total number of these factors in place at baseline correlated with numbers trained in the first year (Rs = 0.77, p = 0.01). Specifically, areas with a healthcare provider to co-ordinate the intervention (p = 0.02); a pool of other volunteers to recruit from (p = 0.02); a contract in place with a commissioned service (p = 0.02); and formal volunteer arrangements (p = 0.03) trained more AHCs. Across all nine areas, fewer volunteers were recruited than the anticipated target of 35 per area. In total, 123 AHCs were trained and gained a Royal Society for Public Health Level 2 qualification. Of these, 95 were lay volunteers from the intervention areas. The majority identified as white (70%), with ages ranging from 18 to 65+, and almost half aged between 41 and 60 (48%). Slightly more women volunteered compared to men (61%). Most AHCs self-reported being in the lower-risk drinking category (66% scored 0–4 on AUDIT-C). Post-training questionnaires suggested that volunteers felt more confident to talk about the harms associated with alcohol and give alcohol-related brief advice than they did pre training (91.4% compared with 79.6%, p < 0.001) and that they felt more confident to raise issues about venues selling alcohol (90.3% compared with 74.2% pre training, p < 0.001). Intervention areas that recorded AHC activity (n = 5) captured 1100 conversations, 251 AUDIT-Cs completed and 65 community events attended. Time/cost diaries suggested that AHCs spent on average 11 minutes having an informal brief advice conversation. Interviews with AHCs on self-reported activity were consistent with time/cost and reflective diaries: AHCs put into practice brief advice conversations more than community action to influence licensing. Interviews with stakeholders indicated several enabling factors to consider when rolling out and embedding an AHC intervention: a clear understanding of the place-based focus; commissioned provider services needing a clear understanding of own roles and the anticipated outcomes; and co-ordinators having the skills and capacity to support inexperienced volunteer groups. The importance of infrastructure was reflected in interviews, recognising that preparing the ground at a hyperlocal level takes time, needing to be well in advance of the formal implementation period. Recruitment strategies need to be multipronged, and stakeholders need to be prepared to provide a sustained period of support for new and existing champions. In the follow-up phase, 12 months post intervention, interviews with stakeholders revealed how the local co-ordinators’ ability to operationalise and sustain CICA was significantly impacted by their lack of capacity to sustain the levels of support, as well as the complexity of skills required of the role. After 12 months, there was strong consistency in findings from follow-up interviews with stakeholders, AHCs and focus groups with members of the public. Communities in Charge of Alcohol’s intended mechanism of action was to reduce alcohol-related harm through secondary prevention interventions; however perceptions of those most in need of an AHC conversation were ‘problem drinkers’ to signpost into tertiary interventions. Quality of local alcohol treatment services was considered to be poor, with multiple barriers to help and information seeking. Conclusions To our knowledge, CICA is the first alcohol-focused champion role of its kind to be investigated and across multiple intervention sites. It is also the first evaluation of a health champion programme that uses methodological triangulation to strengthen inferences about effectiveness. There was evidence that lay people trained as AHCs were able to have conversations with members of their communities who were consuming alcohol at higher risk levels, demonstrating a level of skill and confidence that can sometimes be absent in health professionals. AHCs, their co-ordinators and the public valued the role. However, significant infrastructure is needed to support an intervention such as CICA, and time is needed to develop and embed a group of volunteers. Despite evidence (from the process evaluation) of the planned activity taking place at local level, we were not able to show that CICA was effective, nor was it cost-effective. There are likely to be two main reasons for this. Firstly, the number of champions trained was too small to have a sufficient number of brief advice conversations to generate any measurable effect at the area level of analysis. Moreover, brief interventions may not directly impact on primary health and crime outcomes at a population level. Secondly, the AHCs were less willing (and had less opportunity) to get involved with alcohol licensing decisions. Since licensing policies theoretically have an impact at an area level, this theoretically would have been more likely to generate a significant area-level effect. The evidence from this study is that communities continue to struggle to influence statutory processes that affect alcohol availability where they live, and further consideration of how to enable increased community engagement is necessary. Recommendations for future research (numbered in priority order): natural experiment evaluations should include methodological triangulation to guard against overinterpretation of spurious results investigate coproducing a community outcomes framework to measure reductions in alcohol harm evaluate whether rebalancing local licensing policy to be community-centred might increase community engagement in the local licensing decision-making process investigate the effectiveness of CICA in areas where indicators of alcohol harm are increasing but are not at the highest level of harm within the local authority a wider range of external contextual factors could be tested in the pre-implementation phase of a complex intervention, incorporating methods to combine factors into mean scores, to test relationships between process indicators and outcomes. Trial registration This trial is registered as ISRCTN81942890. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 15/129/03) and is published in full in Public Health Research; Vol. 12, No. 9. See the NIHR Funding and Awards website for further award information.
Keywords
