Health and Social Care Delivery Research (Apr 2024)
Evaluation of different models of general practitioners working in or alongside emergency departments: a mixed-methods realist evaluation
Abstract
Background Emergency healthcare services are under intense pressure to meet increasing patient demands. Many patients presenting to emergency departments could be managed by general practitioners in general practitioner–emergency department service models. Objectives To evaluate the effectiveness, safety, patient experience and system implications of the different general practitioner–emergency department models. Design Mixed-methods realist evaluation. Methods Phase 1 (2017–8), to understand current practice: rapid realist literature review, national survey and follow-up key informant interviews, national stakeholder event and safety data analysis. Phase 2 (2018–21), to collect and analyse qualitative (observations, interviews) and quantitative data (time series analysis); cost–consequences analysis of routine data; and case site data for ‘marker condition’ analysis from a purposive sample of 13 case sites in England and Wales. Phase 3 (2021–2), to conduct mixed-methods analysis for programme theory and toolkit development. Results General practitioners commonly work in emergency departments, but delivery models vary widely in terms of the scope of the general practitioner role and the scale of the general practitioner service. We developed a taxonomy to describe general practitioner–emergency department service models (Integrated with the emergency department service, Parallel within the emergency department, Outside the emergency department on the hospital site) and present a programme theory as principal output of the study to describe how these service models were observed to operate. Routine data were of variable quality, limiting our analysis. Time series analysis demonstrated trends across intervention sites for: increased time spent in the emergency department; increased emergency department attendances and reattendances; and mixed results for hospital admissions. Evidence on patient experience was limited but broadly supportive; we identified department-level processes to optimise the safety of general practitioner–emergency department models. Limitations The quality, heterogeneity and extent of routine emergency department data collection during the study period limited the conclusions. Recruitment was limited by criteria for case sites (time series requirements) and individual patients (with ‘marker conditions’). Pandemic and other pressures limited data collection for marker condition analysis. Data collected and analysed were pre pandemic; new approaches such as ‘telephone first’ and their relevance to our findings remains unexplored. Conclusion Findings suggest that general practitioner–emergency department service models do not meet the aim of reducing the overall emergency department waiting times and improving patient flow with limited evidence of cost savings. Qualitative data indicated that general practitioners were often valued as members of the wider emergency department team. We have developed a toolkit, based on our findings, to provide guidance for implementing and delivering general practitioner–emergency department services. Future work The emergency care data set has since been introduced across England to help standardise data collection to facilitate further research. We would advocate the systematic capture of patient experience measures and patient-reported outcome measures as part of routine care. More could be done to support the development of the general practitioner in emergency department role, including a core set of competencies and governance structure, to reflect the different general practitioner–emergency department models and to evaluate the effectiveness and cost effectiveness to guide future policy. Study registration This study is registered as PROSPERO CRD42017069741. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 15/145/04) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 10. See the NIHR Funding and Awards website for further award information. Plain language summary Hospital emergency departments are under huge pressure. Patients are waiting many hours to be seen, some with problems that general practitioners could deal with. To reduce waiting times and improve patient care, arrangements have been put in place for general practitioners to work in or alongside emergency departments (general practitioner–emergency department models). We studied the different ways of working to find out what works well, how and for whom. We brought together a lot of information. We reviewed existing evidence, sent out surveys to 184 emergency departments, spent time in the emergency departments observing how they operated and interviewing 106 staff in 13 hospitals and 24 patients who visited those emergency departments. We also looked at statistical information recorded by hospitals. Two public contributors were involved from the beginning, and we held two stakeholder events to ensure the relevance of our research to professionals and patients. Getting reliable figures to compare the various general practitioner–emergency department set-ups (inside, parallel to or outside the emergency department) was difficult. Our findings suggest that over time more people are coming to emergency departments and overall waiting times did not generally improve due to general practitioner–emergency department models. Evidence that general practitioners might admit fewer patients to hospital was mixed, with limited findings of cost savings. Patients were generally supportive of the care they received, although we could not speak to as many patients as we planned. The skills and experience of general practitioners were often valued as members of the wider emergency department team. We identified how the care provided was kept safe with: strong leaders, good communication between different types of staff, highly trained and experienced nurses responsible for streaming and specific training for general practitioners on how they were expected to work. We have produced a guide to help professionals develop and improve general practitioner–emergency department services and we have written easy-to-read summaries of all the articles we published. Scientific summary Background Emergency healthcare services are under intense pressure to meet the increasing patient demands. Many patients presenting to emergency departments (EDs) could be managed by general practitioners (GPs). We aimed to evaluate the effectiveness, safety, patient experience and system implications of the different models of GPs working in or alongside EDs (GP-ED). Objectives Identify which models are in place. Describe how the models work. Describe the outcomes of each model. Explain the relationships between contexts (C), mechanisms (M) and outcomes (O) to develop a programme theory to help inform service delivery in other settings. Design We conducted a mixed-methods realist evaluation to describe what works, for whom, in what circumstances and how over three phases. Phase 1: understanding current practice and which models are in place Rapid realist review Method We conducted a rapid realist review to develop initial theories about how different GP-ED models operate, supported by our co-applicants as an expert reference group. Results A total of 96 articles contributed to initial theories: how decisions on streaming patients to GP services are influenced; the role of GPs in EDs; patient satisfaction and safety concerns; the risk of provider-induced demand in highly visible services; and whether these services represent value for money. National survey and follow-up clinical director interviews Method We sent an online survey to 184 clinical directors (CDs) from Type 1 EDs in England and Wales to gather information on aims, implementation and delivering the GP-ED services. Thirty CDs who worked in services implemented since 2010 (a range of locations, models and sizes) were invited to take part in semistructured telephone interviews to explore: service operation, perceived successes and challenges. Results We received 77 responses out of 184 invitees (41.3%); 51 (66%) respondents used a GP-ED model. We interviewed 21 CDs. Findings were used to characterise and categorise GP-ED models to create a taxonomy, inform case study site selection and further theory development. National stakeholder event Method We held a National Stakeholder event in Bristol in 2018 (n = 48 attendees), to present findings from the review and survey, gather stakeholder feedback on a taxonomy of GP-ED models (in collaboration with researchers from the GP-ED team at the University of the West of England) and identify ‘marker conditions’ to guide data collection in Phase 2. Results The taxonomy included a description of the ‘form’ of the GP-ED models (inside the ED: Integrated with ED service delivery or a separate Parallel service; or Outside the ED: on or off the hospital site) and a description of how different constructs influence how the service may ‘function’ – as a traditional GP service or emergency medicine service. Five marker conditions were selected, including: child with a fever, cough/shortness of breath, abdominal pain, back pain and chest pain. (We added ‘headache’ as a further condition in Phase 2 of the study to enhance patient interview recruitment.) National patient safety data collection and analysis Method We searched Coroners’ Reports to Prevent Future Deaths (2013–8) and National Reporting and Learning System (NRLS) patient safety incident reports (2005–15) relating to diagnostic errors in GP-ED services. We coded these using the Patient Safety Classification System (PISA) framework and developed initial safety theories to inform data collection in Phase 2. Results We identified 9 relevant Coroners’ Reports and 217 NRLS reports. Initial theories were developed around: difficulty with triage and streaming processes; errors in clinical decision-making; and inadequate referral pathways and communication between services. Phase 2: case study mixed-methods data collection and analysis to describe how the general practitioner–emergency department models work and outcomes Qualitative data analysis Method We selected 13 case study sites reflecting different GP-ED models (3 Integrated, 4 Parallel, 3 Outside and 3 with no GP-ED model), different locations, sizes of ED and experience delivering the service. We visited all sites (2–4 days) and conducted observations and realist interviews with staff to refine initial theories. We requested local patient safety incident reports related to the GP-ED model and invited patients presenting with marker conditions for interviews to describe their experience. We analysed data from the multiple data sources and applied knowledge from conceptual frameworks and formal theories to refine our initial theories. We then mapped context–mechanism–outcome configurations against different GP-ED models to compare across different types of service. We presented findings at a second national event on 3 December 2019 (n = 70 attendees) for stakeholder feedback. Results Across all sites, we interviewed 106 staff members, collected 14 anonymised local patient safety incident reports, invited 748 patients to take part in telephone interviews and subsequently interviewed 24 patients. Theories were refined to describe: how streaming processes were influenced by nurse experience, guidance and overall operational and strategic management; how GPs found working in EDs influenced their clinical decision-making to maintain a usual GP approach or adopt an ED clinician approach, mitigating safety risks; factors that facilitate teamwork and communication between GP-ED services and the ED; how separate, visible GP-ED services are perceived to contribute to provider-induced demand; and patients’ expectations and experiences of using services. Routine data analysis Method We obtained patient-level data relating to ED attendances and subsequent hospital admissions from hospital episode statistics accident and emergency and admitted patient care (APC) data sets, via NHS Digital for study sites located in England; and from emergency department data set and patient episode database for Wales data sets (via secure anonymised information linkage) for one study site located in Wales. We summarised the attendance-level data as time series (per site, aggregating data for each study fortnight) for the following ED variables: Attendances; Reattendances; Hospital admissions (defined by patient record appearing in the APC data set); Investigations and Treatments; Time in ED; length of stay (LOS) of hospital admission. For all variables, we used standard time series analysis methods to assess the nature and extent of linear trends and seasonality in data before and after an intervention point at intervention sites. Results Data from 1 October 2010 to 30 September 2018 were eligible for inclusion and were of variable quality, tending to decrease as the level of integration between to GP-ED model and the ED decreased. We are more confident in the interpretations made about some outcomes (attendances, reattendances, admissions, LOS) than others (investigations, treatments, average time in ED) due to data quality. In general, at the GP-ED model sites, attendances increased over time. Reattendances within 28 days in the post-intervention period increased over that in the pre-intervention period in eight of the nine sites assessed. Overall, intervention sites also demonstrated a consistent trend of increasing average time in the ED across both the pre- and post-intervention phases. A mixed picture was seen in relation to hospital admissions in the Integrated and Parallel sites, with data quality issues at the Outside sites. Average length of hospital stay showed a mixed picture over all intervention sites. Cost–consequences analysis Method We conducted a cost–consequences analysis (CCA) using the routine data described above to consider the costs and resource consequences resulting from, or associated with, the use of the GP-ED model types compared to control sites. Results Negligible incremental ED attendance cost differences were observed between the model types and the control with an extremely small increased cost [£0.70, standard error (SE) £0.07] observed when looking at all models combined over control, though a saving of £72 (SE £3.18) when looking at inpatient admissions. When looking at ED visits by individual models, the Integrated model saw a small cost saving (£8.73, SE £0.07), but small cost increases were seen in the Parallel model (£9.51, SE £0.08) and the Outside model (£16.30, SE £0.16). With respect to inpatient admissions, all models saw small cost savings [£25 (SE £4.16) to £215 (SE £9.05)] with the Outside model showing the largest saving (£215, SE £9.05). Marker condition analysis Method We used a CCA as above to evaluate the management of six marker conditions at a Parallel GP-ED model as a case study site (the only site with data available during the pandemic period). Anonymised patient-level data were used to compare proportions of patients admitted to hospital (Pearson’s chi-squared test), and times (non-parametric Mann–Whitney U-test) for ED clinicians and GPs. A backward enter stepwise approach was taken with non-statistically significant variables (p > 0.05) removed. Results Emergency department clinicians saw most patients and those patients categorised as being more severely unwell. Across all marker conditions, GPs managed patients more quickly – the main driver being the time taken from treatment to discharge. Across all marker conditions, the trend was for GPs to admit fewer patients to inpatient care, compared with ED clinicians. This trend was more evident (and for some reaching statistical significance) among those requiring the lowest levels of intervention. Reattendance rates at 7 days were equal for both GPs and ED clinicians. Phase 3: mixed-method analysis, programme theory and toolkit development Mixed-methods analyses Method We conducted mixed-methods analyses to further refine our theories through two approaches: firstly, to identify questions raised through the qualitative data analysis to support, refute or refine through quantitative data analyses; then to identify noteworthy findings from the statistical analysis and cross-check with both the qualitative and marker conditions analyses. Results Streaming The marker condition and qualitative data analysis indicated the potential for GPs to improve the flow of the least unwell patients through the ED. However, at the whole ED level examined by the routine data, the analysis demonstrated increased time in the ED for patients at most sites. Therefore, our theories related to streaming and patient flow have limited support from quantitative data. General practitioner role Marker condition data from a single Parallel case study site showed that patients spent less time in the department and were less likely to be admitted, with no difference in reattendance rates at 7 days. Data on investigation use were of poor quality and not included in the analysis. Routine data for all intervention sites and three control sites also showed poor quality of investigation data. Any influence on time in the department and hospital admissions described in the marker condition data was not seen consistently in routine data analysis. While the marker conditions analysis supports the theory that a GP approach in an ED setting can be different to an ED clinician approach, it remains unclear whether this is due to individual clinicians’ management behaviours or service level differences in ways of working. The quantitative data did not identify particular GP-ED models associated with favourable outcomes potentially attributable to facilitating the ‘GP role’. Safety The routine data indicated increasing rates of reattendance at 28 days across most intervention and control sites. This may reflect changes in levels of morbidity among the population, public behaviours or service configurations (including availability of services elsewhere, notably in-hours primary care) rather than necessarily representing a change in the quality of the care provided at the ED. The marker condition data suggested that GP care in the ED appeared to be as safe as ED clinician care using reattendances within 7 days as a proxy. We have not identified data to develop our theory about strength of communication and teamworking within the ED. Patient experience While the marker conditions analysis indicated that patients attending ED who required the fewest investigations and treatments might be more satisfied by being seen by a GP due to a shorter ED stay, we could not identify any particular GP-ED model that appeared most likely to consistently decrease department stay. Demand Our theory about provider-induced demand in distinct and visible services is not supported by the data on ED attendances over the study period, but our qualitative findings suggest that there is a perceived additional demand for primary care at an ED in services where the primary care service is visible, easily accessible and more well known in the local area. Programme theory development We developed a programme theory, as the principal out of the research, presented as the patient journey through the three GP-ED models, to highlight similarities and differences. We recognise that each site had unique characteristics, so this represents a high-level summary of key features, rather than a description of every possible variation. At Integrated models, the streaming process is less influential. The ‘invisibility’ of the GP service means its impact on patient expectations is also more limited, but it allows for GPs to take on supervision of junior ED doctors, described positively at some sites. At Outside models, the process of ensuring that the right patient saw the GP was more complex, and open to potential problems. Their high visibility and accessibility were likely to have a greater impact on patients’ expectations and experiences. At these sites, GPs often took on a supervisory role for a wider primary care team. Parallel models showed the most variation in the way services were set up and the clarity of the GP role: some more like Integrated models others, Outside models. Toolkit We worked with study co-applicants and a stakeholder group of CDs to translate the research findings and programme theory into guidance for implementing and delivering GP-ED services. Some key points include: A culture including strong clinical leadership, encouraging mutual respect, interprofessional communication and teamworking is essential. To ensure that patients are efficiently and safely streamed, nurses should be highly trained and experienced; streaming pathways/protocols must be clear with senior oversight and include quality improvement systems. Training for GPs based on the type of GP-ED model in use and their intended role. Patients need to be informed about reasons for being streamed to different clinicians to help manage expectations. Conclusion General practitioners commonly work in EDs, but delivery models vary widely in terms of the scope of the GP role and the scale of the GP service. We developed a taxonomy to describe GP-ED models (Integrated, Parallel, Outside) and present a programme theory to describe how these models were observed to operate. Routine data were heterogeneous and of variable quality limiting analysis, including verification of the proposed theories, but trends were noted across intervention sites for: increased time spent in the ED, increased ED attendances and reattendances and mixed findings for hospital admissions. Findings suggest that GP-ED service models do not meet the aim of reducing ED waiting times and improving patient flow, with limited evidence of cost savings. Qualitative data indicated that GPs were often valued as members of the wider ED team, with significant appetite to utilise their specific skillset to provide care in the ED. We developed a toolkit, based on our findings, to provide guidance for implementing and delivering GP-ED services. Study registration This study is registered as PROSPERO CRD42017069741. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 15/145/04) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 10. See the NIHR Funding and Awards website for further award information.
Keywords
