Health and Social Care Delivery Research (Sep 2024)
Investigating innovations in outpatient services: a mixed-methods rapid evaluation
Abstract
Background Within outpatient services, a broad range of innovations are being pursued to better manage care and reduce unnecessary appointments. One of the least-studied innovations is Patient-Initiated Follow-Up, which allows patients to book appointments if and when they need them, rather than follow a standard schedule. Objectives To use routine national hospital data to identify innovations in outpatient services implemented, in recent years, within the National Health Service in England. To carry out a rapid mixed-methods evaluation of the implementation and impact of Patient-Initiated Follow-Up. Methods The project was carried out in four sequential workstreams: (1) a rapid scoping review of outpatient innovations; (2) the application of indicator saturation methodology for scanning national patient-level data to identify potentially successful local interventions; (3) interviews with hospitals identified in workstream 2; and (4) a rapid mixed-methods evaluation of Patient-Initiated Follow-Up. The evaluation of Patient-Initiated Follow-Up comprised an evidence review, interviews with 36 clinical and operational staff at 5 National Health Service acute trusts, a workshop with staff from 13 National Health Service acute trusts, interviews with four patients, analysis of national and local data, and development of an evaluation guide. Results Using indicator saturation, we identified nine services with notable changes in follow-up to first attendance ratios. Of three sites interviewed, two queried the data findings and one attributed the change to a clinical assessment service. Models of Patient-Initiated Follow-Up varied widely between hospital and clinical specialty, with a significant degree of variation in the approach to patient selection, patient monitoring and discharge. The success of implementation was dependent on several factors, for example, clinical condition, staff capacity and information technology systems. From the analysis of national data, we found evidence of an association between greater use of Patient-Initiated Follow-Up and a lower frequency of outpatient attendance within 15 out of 29 specialties and higher frequency of outpatient attendance within 7 specialties. Four specialties had less frequent emergency department visits associated with increasing Patient-Initiated Follow-Up rates. Patient-Initiated Follow-Up was viewed by staff and the few patients we interviewed as a positive intervention, although there was varied impact on individual staff roles and workload. It is important that sites and services undertake their own evaluations of Patient-Initiated Follow-Up. To this end we have developed an evaluation guide to support trusts with data collection and methods. Limitations The Patient-Initiated Follow-Up evaluation was affected by a lack of patient-level data showing who is on a Patient-Initiated Follow-Up pathway. Engagement with local services was also challenging, given the pressures facing sites and staff. Patient recruitment was low, which affected the ability to understand experiences of patients directly. Conclusions The study provides useful insights into the evolving national outpatient transformation policy and for local practice. Patient-Initiated Follow-Up is often perceived as a positive intervention for staff and patients, but the impact on individual outcomes, health inequalities, wider patient experience, workload and capacity is still uncertain. Future research Further research should include patient-level analysis to determine clinical outcomes for individual patients on Patient-Initiated Follow-Up and health inequalities, and more extensive investigation of patient experiences. Study registration This study is registered with the Research Registry (UIN: researchregistry8864). Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 16/138/17) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 38. See the NIHR Funding and Awards website for further award information. Plain language summary When someone visits hospital for an operation or an ongoing condition, they are given follow-up appointments at clinics, often after 6 months. The National Health Service thinks that many of these appointments are not necessary because they are not useful to patients. Also, outside fixed appointments, patients are not always being seen when they are most in need. Hospitals have been testing new ways to improve services. We looked at hospital data and discussed interesting findings with hospitals themselves to see if we could find approaches that worked. We then looked at one new approach called Patient-Initiated Follow-Up. Patients using Patient-Initiated Follow-Up can book appointments when they are needed, rather than at a pre-planned time. We explored how Patient-Initiated Follow-Up was being used in hospitals, what effect it was having and what patients and National Health Service staff thought about it. We studied hospital data and interviewed patients and National Health Service staff. We found that Patient-Initiated Follow-Up works differently depending on the hospital and the patient’s condition. Patient-Initiated Follow-Up is most frequently used for patients needing short-term follow-up, such as after an operation. It is also starting to be used for patients with long-term chronic conditions. National Health Service staff think that Patient-Initiated Follow-Up can benefit patients, although some may find it easier to use than others. Patients appear to like Patient-Initiated Follow-Up, but some still prefer to let the hospital schedule appointments. From data it appears that for some conditions, where more patients use Patient-Initiated Follow-Up, fewer follow-up visits are required. For a few conditions, there is evidence of fewer emergency department visits, but the overall impact is small. We interviewed staff from a small number of hospitals and four patients, so what we found may not apply across the National Health Service. We also developed a guide to help hospitals evaluate the success of their own Patient-Initiated Follow-Up services. Scientific summary Background The number of outpatient attendances in the UK has increased in recent years, outstripping population growth and unmatched by comparable increases in workforce or capacity. Improving the value of outpatient care has therefore become a key priority for the NHS in England, which aimed to reduce follow-up outpatient attendances by 25% from 2019 to 2020 levels by March 2024. Across the outpatient care pathway, a broad range of innovations are being pursued to better manage outpatient care and reduce unnecessary appointments. The NHS also aims to link such innovations to a more patient-centred service by making better use of primary, secondary, and community services and helping people to manage their own health. However, there is limited understanding of which interventions are effective and what factors contribute to their success. We aimed to devise a methodology for using data to identify potentially successful innovations in outpatient services and to select up to two for more in-depth evaluation. The innovation we chose was Patient-Initiated Follow-Up (PIFU) because it was not well studied, was becoming widely implemented across the NHS and NHS England were interested in it being evaluated, with findings informing the evolving outpatient transformation policy. With PIFU, patients can make their own follow-up appointments when they need them rather than be called back at pre-planned times (e.g. every 6 months). The intention is that follow-up appointments become more clinically valuable by increasing the chances that patients are seen at the right time. However, when we began the evaluation, it was little known what impact PIFU would have on patient outcomes, service resources, and patient and staff experience. Objectives This study aimed to identify innovations in outpatient services implemented in recent years in the English NHS and to carry out a rapid evaluation of PIFU. The study was divided into four sequential workstreams with the following aims: To understand the scope and breadth of interventions being pursued to improve efficiency in outpatient service delivery, and to understand key evidence gaps and research needs. To identify trusts and/or specialties where there is quantitative evidence of a positive change to outpatient activity, for example, a reduction in the numbers of attendances or a substitution between different modes of attendance (e.g. from face-to-face consultation to teleconsultation). To undertake interviews of selected trusts and specialties identified in workstream 2. To investigate whether changes in their outpatient activity were the result of specific innovations in care management. To conduct a mixed-methods evaluation of PIFU, considering its implementation, impact, and the experiences of patients and staff. Methods The project was supported throughout by an advisory group and patient and public involvement representatives. The approaches we used within each workstream were as follows: Workstream 1: scoping review of innovations We adopted an evidence-mapping approach and a literature search to understand the extent to which different outpatient service innovations have been studied and the potential benefits they evaluated. Data were synthesised narratively by type of intervention and their anticipated benefit using a pre-specified framework. The results of this review would inform the choice of innovation to evaluate in workstream 4. Workstream 2: identifying innovations using routine hospital data We developed a methodology to scan routine hospital outpatient data to identify notable positive changes in activity measures. We applied a modelling approach called indicator saturation (IS) to multiple time series extracted from the data. These time series covered a range of clinical specialties within hospital trusts. Four outpatient activity measures were selected where changes might have indicated the impact of a service change or innovation. We built a series of metrics to define the scale and other characteristics of positive changes detected within the time series. Combining these, we ranked all time series to identify those most likely to show the impact of an active service change, or innovation. While this work began with a general focus (with any type of service change in scope), we adapted the methods to support the evaluation of PIFU by focusing on a single activity measure: the ratio of follow-up to first attendances. Workstream 3: interviews with trusts We invited services to be interviewed about recent notable falls in the ratio of follow-up to first attendances. Interviewees were presented with details of the findings and were asked to reflect on any service changes that may have driven the changes. Workstream 4: evaluation of Patient-Initiated Follow-Up Workstream 4 was a rapid mixed-methods evaluation of PIFU conducted across two phases with a preliminary scoping review intended to both provide a context for and inform the rest of the evaluation. For the review, we adopted an evidence-mapping approach to understand what was already known about the impact of personalised follow-up initiatives on service use, patient experience, and outcomes. We included studies published between January 2015 and June 2022. Phase 1 was a 4-month evaluation using national data sets alongside interviews with local and national stakeholders. The qualitative aspects focused on how PIFU was being implemented within specialties where it was most established. We analysed national data to assess the impact of implementing PIFU on a range of outcomes relating to a patient’s use of hospital services. Phase 2 built on the insights from phase 1 and consisted of interviews with patients and staff, and a workshop to discuss factors affecting PIFU uptake. Compared to phase 1, the purpose was to investigate issues in greater depth and across a wider range of specialties and conditions. We also undertook further quantitative analysis of both local and national data. Finally, we developed an evaluation guide to support ongoing evaluation of local PIFU services. For both phases we adopted a sampling framework for selecting study sites to achieve a balance across a range of variables relating to the type of hospital, and the local geography and population. Staff for interviews were selected through a purposive approach which evolved into convenience sampling due to the availability of staff. Results Scoping review of innovations The scoping review found a variety of innovations in outpatient care that have been evaluated within different health systems. Some of these include changing how appointments are delivered (e.g. teleconsultations), new administration and support systems, specialised support for primary care and personalised follow-up. The purposes of these innovations were grouped into three main areas, for making booking systems more efficient, for optimising use of resources and for improving the clinical value of follow-up appointments. Virtual appointments and administration and scheduling systems are the most evaluated innovations, and only a few studies investigated personalised follow-up. Identifying and investigating innovations using routine hospital data We successfully applied IS modelling to over 20,000 time series of outpatient activity measures. We selected nine services for further investigation and invited relevant staff for interview. Of these, three were interviewed. Two services did not recognise the data findings and had no explanation for the apparent improvements we found. The third recognised the findings and attributed the change to a clinical assessment service. A fourth (not interviewed) identified the change as having been connected to a change in medication use. Evaluation of Patient-Initiated Follow-Up Evidence review We found little evidence on how PIFU is being implemented in the NHS and its impact on staff, patients, and services. Where it had been researched, there was some evidence that PIFU might result in fewer overall outpatient appointments compared with fixed appointment schedules, although results were variable and most studies were of low quality. There was little evidence that PIFU affects patient satisfaction, quality of life, or clinical outcomes. The impact on the wider health services and costs were not well studied. Sites and participants We conducted semistructured interviews with 36 clinical and operational staff based in 5 case study sites and 4 patients. The workshop included 22 staff from a further 13 NHS trusts. Implementation of Patient-Initiated Follow-Up By early 2023, approximately 180,000 patients a month were recorded as being put onto a PIFU pathway. PIFU is most commonly being used in short-term pathways (e.g. physiotherapy or following surgery), although there are examples where it is being used for people with long-term conditions. Models of PIFU vary widely across trusts and clinical area, with a significant degree of variation in patient selection, monitoring, and discharge. The nature of a patient’s condition was a key factor in how it was implemented. Where PIFU had been implemented successfully, enablers included conditions where symptoms and deterioration were easy to identify, clinical engagement, supporting guidance, champions, dedicated staff capacity and flexible recording systems. Barriers to implementation included patients not being aware they were on PIFU, staff resistance, competing priorities and limited capacity to dedicate to PIFU, a lack of engagement with primary care and challenges amending electronic patient record systems to record PIFU activity. Staff saw several opportunities for PIFU, including supporting their service to become more efficient and patient-centred. They also saw opportunities for wider adoption of the innovation, and better use of technology to facilitate delivery. However, staff also pointed to several risks with PIFU, especially where patients do not or cannot access the service when they should. They also raised risks for service capacity within and beyond outpatient departments. Impact of Patient-Initiated Follow-Up on outpatient and emergency department attendance Increasing PIFU rates were associated with less frequent outpatient attendance and rates of patient non-attendance [‘did not attend’ (DNA)], particularly within certain clinical specialties. However, within some specialties increased PIFU rates were associated with more frequent visits. This complemented findings from interviews with staff and from the workshop in that the variety of ways PIFU is implemented can lead to different impacts. We found no practically significant association between PIFU rates and frequency of emergency department (ED) visits overall (results were statistically significant, but of negligible effect size), but a small number of specialties appeared to have increasing PIFU rates associated with less frequent ED visits. Staff at study sites described limitations of their PIFU data for monitoring their outcomes. Moreover, existing data is not currently able to capture wider consequences, such as the impact on primary care. The study’s findings need to be interpreted with the understanding that we used aggregated PIFU data as available data does not currently record which patients are on PIFU pathways. There is also some uncertainty about the completeness of the available PIFU data. Health inequalities There was limited understanding of the impact of PIFU on different patient groups, and staff recognised this as a concern. Digital exclusion, demographic characteristics, socioeconomic status and patient characteristics were all thought to be relevant to how patients engage with or are impacted by PIFU. Some staff were concerned that PIFU could exacerbate inequalities because those with greater individual motivation and ability to advocate for themselves would be more able or willing to initiate contact when they needed. However, some services have used PIFU for engaging with patients who were more likely to miss appointments by enabling them to contact the service at a time that was convenient for them. Staff experiences of Patient-Initiated Follow-Up In most cases, PIFU was viewed by staff as a positive intervention to support patient autonomy and self-management and ensure that time was directed towards those patients with the greatest clinical need. Views on the impact of PIFU on individual staff roles and workload were mixed, and dependent on the extent to which PIFU was used routinely within the service, the nature of the pathway and extent to which PIFU was considered a departure from previous ways of working. Patient experiences of Patient-Initiated Follow-Up Across the case study sites, limited activity had taken place to capture formal feedback from patients, although staff reported that patients were positive about PIFU as an approach and the support they received. Reasons for patients declining PIFU included a preference for regular interaction, a desire to stick to their routine, and a concern that they would be unable to get an appointment when needed. Patients we spoke to were positive about their experiences and liked having the option to contact a specialist when they needed to. Enablers to patient engagement include clear avenues of access and support, communication and ensuring patients do not feel abandoned by the service. Barriers to patient engagement included a lack of awareness and understanding about PIFU, the wider context on access to services and factors related to specific conditions. Evaluation guide We developed a guide to support teams to evaluate the outcomes and impacts of their PIFU services at a specialty level, thus helping address research gaps and encouraging shared learning. Limitations We were unable to develop a reliable methodology to rank time series across multiple outpatient activity measures to identify those most likely to show the impact of a service change. The final selection of changes in performance most likely to be connected to implementation of PIFU required more manual researcher input than anticipated, and the results of a small number of interviews with selected hospitals were variable. Conducting a rapid study of an intervention which was still being rolled out alongside other interventions, where data were limited and when there were significant pressures on NHS staff capacity, introduced considerable challenges. Consequently, the sample sizes for hospital sites, staff and patients were small. Owing to this, the qualitative findings cannot be assumed to be representative of the national picture, although they provide valuable insights into the use of PIFU in outpatient services. Due to the lack of quantitative data on outcomes, we have focused the analyses of the impact of PIFU on changes in outpatient activity and ED attendance and have not been able to use more patient-focused measures. Conclusions With many initiatives being undertaken in NHS hospitals and for different clinical conditions, distinguishing those that provide benefits for outpatient care can be a formidable task. We have found that the implementation of PIFU is associated with significant reductions in the frequency of outpatient attendances and DNAs in some specialties and significant increases in a smaller number of others. However, we do not know if these relationships are all due to PIFU and, because of the available data, have not been able to measure impacts on those specific individuals who are put onto PIFU pathways. Many of the findings have implications for the organisation of outpatient services at a national and local level. These include ensuring PIFU guidance is tailored to specialties or conditions and ensuring communication about PIFU, and its purpose is clear, consistent and accessible to both staff and patients. PIFU is generally perceived as a positive intervention for staff and patients but the impact on individual outcomes, health inequalities, wider patient experience, workload and capacity is still uncertain. Recommendations for research Data collection should include more meaningful patient outcomes and health inequalities alongside patient experiences and impact on staff workload. More research should be conducted on the views and experiences of patients and staff relating to PIFU, particularly with staff in clinical specialties not covered by our own evaluation. More research is needed to understand how PIFU interacts with other outpatient interventions and the wider health and social care system. National data sets could be better used to scan for exceptional changes in relation to outpatient care, combining this with fieldwork to identify whether these correspond to specific innovations. Study registration This study is registered with the Research Registry (UIN: researchregistry8864). Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 16/138/17) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 38. See the NIHR Funding and Awards website for further award information.
Keywords
