Introducing artificial intelligence tools into clinical and non-clinical workflows, Evidence from the NHS Breast Screening Programme of staff and general population perceptions of and trust in the use of AI in this context and the implications for further testing and implementation.
Niamh Lennox-Chhugani(1), Jonathan James(2), Bernadette Trzcinski(3)
(1)TaoHealth Research & Implementation, (2)Nottingham Breast Institute, Nottingham University Hospitals NHS Trust, (3)United Lincolnshire Hospitals NHS Trust
Background: The East Midlands Radiology Consortium (EMRAD) is a partnership of seven NHS trusts spread over 11 hospitals, covering more than five million patients in the East Midlands of England. In 2018, EMRAD formed a partnership with two UK-based Artificial Intelligence (AI) companies, Faculty and Kheiron Medical, to help develop, test and - ultimately - deploy AI tools in the breast cancer screening programme in the East Midlands as part of wave two NHS Test Beds programme. The project aims to improve and optimise clinical service capacity, to enhance patient care at significant scale and to increase NHS confidence in the utilisation of innovative machine learning tools. The successful implementation of the technology rests not only on its clinical effectiveness but also on the attitudes of key adopters and organisational readiness (Greenhalgh et al, 2017).
Three of the questions the evaluation of the project sets out to answer is:
1. What are the perceptions of staff in the NHSBSP of the potential use of AI tools to support operations and clinical decisions today?
2. Do these attitudes change when staff are involved in testing new AI tools in the NHSBSP?
3. What are the perceptions of women in the general public of the potential use of AI tools to support operations and clinical decisions within the NHSBSP?
Relatively little has been written on this topic to date and there has been even less empirical research exploring attitudes to AI in healthcare. This Health Research Authority approved study addresses this gap.
Methods: Clinicians and breast screening mangers are the frontline staff impacted by the use of artificial intelligence tools that are being developed in this NHS Test Bed. We are currently surveying, via an online survey tool, clinicians and service managers during November 2019 – January 2020 to explore attitudes towards artificial intelligence and its use in the breast screening programme. We will administer the survey to the same participants again in Summer 2020 when the AI tools are being prospectively piloted to determine whether staff attitudes towards artificial intelligence changes as a consequence of direct experience of AI tools. Our sample includes clinicians and service managers at Nottingham University’s Hospital NHS Trust, United Lincolnshire Hospitals NHS Trust, Northamptonshire Healthcare NHS Foundation Trust and Sherwood Forest Foundation Trust, all partners in the emrad consortium.
We are also conducting a survey focusing on the attitudes of the general public to the use if AI tools in NHSBSP. We are surveying all female employees of the four trusts in the same period as a representative sample of the genera public. The national health service breast screening programme is targeted towards women between the ages of 50 and 70 years old. As such, this is a primary population that we are targeting but we also want to understand the attitudes of younger women as they will become users of the service in the future. We will be conducting targeted focus groups with populations that are under-represented in the survey (women over 60 years and women in full or part-time paid emloyment).
Work to date and expected outputs by July 2020:
We have conducted a qualitative interim evaluation of the process of the project to date using the NASSS framework (Greenhalgh et al 2017) and are in the process collecting survey data from our sample populations. We will be in a position to share the initial findings from the quantitative and qualitative analysis of the surveys at the HSRUK Conference in July 2020.
Implications: The results can be used to inform the design, implementation and regulation of emerging AI tools in the context of the NHS.
Predictors of Hearing Help-Seeking among Older Adults in England and Implications for Health Policy Strategies in Primary Care
Dalia Tsimpida(1), Evangelos Kontopantelis(2), Darren Ashcroft(3), Maria Panagioti(4)
(1)Centre for Primary Care and Health Services Research, The University of Manchester, (2)Institute for Health Policy and Organisation (IHPO), (3)NIHR Greater Manchester Patient Safety Translational Research Centre (PSTRC), (4)Centre for Primary Care and Health Services Research, NIHR Greater Manchester Patient Safety Translational Research Centre (PSTRC)
Background Hearing loss is a major public health issue that affects over 9 million people in England. Traditionally, people with hearing difficulties will present to their GP to seek advice and investigation. Many of these people will be referred to Secondary Care Audiology or ENT for assessment and management. However, as hearing loss almost always develops gradually, people do not see it as a dramatic health problem requiring urgent intervention. In the absence of a HL screening program in England, It is therefore important to examine whether the self-identification of hearing problems is accurate, as it affects the initiation of hearing help-seeking.
Method Cross-sectional analysis of participants aged 50-89 years old from a representative sample of adults aged 50 years and over living in England (the English Longitudinal Study of Ageing, Wave 7). Our cohort was composed of 8,529 individuals that had assessment in their hearing by both self-reported measures and consented for assessment by a qualified nurse via a hearing screening device (HearCheck™ Screener). Hearing loss was defined as >35dB HL at 3.0 kHz, in the better-hearing ear. Multiple logistic regression models examined whether the self-reported measures of hearing -including hearing in background noise- are valid in comparison to objective measures of hearing and which the predictors of the potential inaccuracies are across different population subgroups of a representative population sample.
Results 30.2% of individuals with HL went undetected by the self-report measure in ELSA. Statistically significant predictors of misreporting hearing difficulties (while they had objectively measured HL >35dBHL at 3.0kHz, in the better-hearing ear) were: female gender (OR 1.97, 95%CI 1.18-3.28), no educational qualifications (OR 1.37, 95%CI 1.26-2.55), routine/manual occupation (OR 1.43, 95%CI 1.28-2.61), tobacco consumption (OR 1.14, 95%CI 1.08-1.90), alcohol intake above the low risk level guidelines (OR 1.13, 95%CI 1.11-2.34), and lack of moderate physical activity (OR 1.25, 95%CI 1.03-1.42). Age was largely associated with misreporting of moderately severe to severe HL; the odds were 5.75 (95%CI 1.17-8.13) higher on those aged 65-74 and 7.08 (95%CI 1.41-9.30) on those aged 75-89 to not report their hearing difficulties compared to those 50-64 years old. Also, socioeconomic indicators such as education (OR 1.95, 95%CI 1.63-6.01) and occupation (OR 2.07, 95%CI 1.78-5.40) along with lifestyle factors such as smoking (OR 1.46, 95%CI 1.25-2.48) and alcohol intake above the low risk level guidelines (OR 1.86, 95%CI 1.67-5.12) were predictors of misreporting moderately severe or severe HL.
Implications The self-identification of hearing difficulties is a major barrier for the initiation of help-seeking, which can affect the referral to ear specialists and the consequent hearing aid uptake. Our study showed that the self-report measurement of HL had limited accuracy and was not sufficiently sensitive to detect HL. In England, up to one third of adults with HL may remain undiagnosed and therefore not referred to ear specialists or given access to hearing aids. Importantly, people belonging in high-risk groups for HL, such as older and less educated people that face socioeconomic inequalities and adopt an unhealthy lifestyle, are the least likely to be accurately identified using self-report measures. These findings provide novel insights into clinical practice and reinforce the importance of an effective and sustainable hearing loss screening strategy in primary care, for the early detection and intervention for HL in older adults.
*This research was funded by the NIHR Manchester Biomedical Research Centre (PhD Studentship). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department
The association of multimorbidity and common mental health disorders with healthcare utilisation by people with type 2 diabetes: a retrospective cohort study using linked electronic health records
Fiona Grimm(1), Meetali Kakad(2), Sarah Deeny(1)
(1)The Health Foundation, (2)Akershus University Hospital, University of Oslo
Background: Type 2 diabetes (T2DM) among the most common long-term health conditions in the UK and a leading cause for morbidity and mortality world-wide. Despite the availability of a range of treatment options, T2DM can be challenging to manage in practice, especially within the context of multiple comorbidities. Common mental health disorders (CMD), such as depression and anxiety, which are more prevalent in people with T2DM, have been shown to be associated with diabetes treatment non-adherence and increased mortality risk. However, there is currently little evidence on how frequently people with T2DM in the UK access healthcare services and how the presence of physical and mental long-term conditions affects glycaemic control, healthcare use and health outcomes. The aim of this study was to describe the comorbidity burden and care utilisation of people with T2DM and to investigate the impact of multimorbidity and CMD on healthcare use and outcomes.
Method: A retrospective cohort study of 68,778 people with prevalent T2DM in the UK, which were identified from practices within the Clinical Practice Research Datalink. Based on published work and expert advice from diabetologists, the presence of T2DM was determined using a multi-step algorithm based on a combination of Read codes and prescriptions for diabetes medication prior to the study start date. Primary care records were used to determine patient characteristics, glycaemic control and the presence of 36 long-term conditions using the Cambridge Multimorbidity Index. Healthcare utilisation across primary and secondary care was quantified from 1 December 2015 over period of up to two years. Multivariate negative binomial regression models were used to examine the effect of CMD and physical conditions on utilisation rates, while controlling for demographic and clinical characteristics.
Results: Most people with T2DM had additional long-term conditions (90%) and having four or more comorbidities was not uncommon (34%). In our cohort, 21.7% had CMD and people with comorbid CMD were also more likely to have other conditions that affect quality of life, such as chronic pain (odds ratio = 3.5, 95% CI = 3.4-3.6). People with T2DM had frequent healthcare contacts, but there was large variation in within the population. Multivariate analysis showed that, while additional physical conditions were associated with higher utilisation (rate ratio [RR] for annual primary care consultations = 1.16, 95% CI = 1.16-1.17), CMD were found to be more strongly associated with healthcare use (RR for annual primary care consultations = 1.29, 95% CI = 1.27-1.30). After adjusting individual comorbidities, CMD remained among the conditions with the strongest association with higher utilisation. On a population level, CMD and chronic pain were also among the most prevalent conditions associated with higher utilisation across healthcare services.
Implications: As T2DM prevalence continues to increase, this study contributes important evidence on the care needs of a growing patient population. Due to the high prevalence of CMD and chronic pain among people with T2DM, the association of these conditions with increased healthcare utilisation highlights that integrated management of comorbidities and mental health support will be central to dealing with growing demand for care and to improve quality of life and health outcomes. Prevalence of T2DM is higher in deprived areas, where people are more likely to have CMD and multiple conditions earlier in life. At the same time, deprived areas see higher pressure on primary care services and more dramatic decreases in GP numbers. Therefore, the results of this study are particularly relevant for planning future care provision aimed at reducing existing health inequalities.
High-Cost High-Need Children and Young People: A Descriptive Analysis
Kathryn Dreyer(1), Sarah Deeny(1), Dougal Hargreaves(2), Thomas Beaney(2)
(1)The Health Foundation, (2)Imperial College London
Background: There is evidence that expenditure is highly concentrated in the minority of patients; however little is known about how health spending is distributed amongst children and young people or how we can identify children at risk of costly health care interventions. The distribution of health care spending is pertinent given the funding pressures facing the English NHS.
This study aims to identify total health spending costs and relative differences across services for the top 5% of high-cost high-need children and young people and to characterise the features of this group. By analysing the distribution and concentration of spending of children and young people it is possible to develop population-specific interventions for this age group.
Methods: This is a retrospective, cross-sectional study of a random sample of 300,000 patients from a large nationally representative research dataset in England comprised of linked administrative primary care and secondary care records (Clinical Practice Research Datalink linked to Hospital Episodes Statistics) for the year 2015/16. Children and young people were defined as all patients under the age of 25. The high-cost high-need group was defined as the top 5% of children and young people based on total costs. Total costs were estimated for primary care visits, emergency department and outpatient attendances, inpatient admissions, and prescriptions.
The primary analysis consists of descriptive statistics that compare costs and utilisation of health care services between the high-cost, high-need group and all other patients, as well as the features of the high-cost, high need patients.
Results: The most common condition in the high-cost high-need group was mental health, with 25% of patients in this group having a mental health diagnosis. 29% of those in the high-cost high-need group in 2015/16 were also classified as high-cost high-need patients in the preceding year. 54% (£17,321,850) of the total yearly health service spending was utilised by the most expensive 5% of children and young people. The proportion of spending on this group was highest for inpatient costs (88%) and lower for costs relating to emergency departments (31%), outpatients (44%), primary care (15%) and medication (38%). Those in the most deprived areas were more likely to be in the high-cost high-need group compared those in the least deprived areas (6% vs 4%).
Implications: Just over half of health service costs for children and young people were spent on only 5% of users. One quarter of children and young people in this group were diagnosed with a mental health condition during the year. 3Proactive strategies are needed to keep children well and reduce the need for costly inpatient care. Further research is needed to understand whether this can be accurately predicted or indeed effectively reduced.
Patient non-attendance at urgent referral appointments for suspected cancer: a mixed methods study of explanations and outcomes
Peter Knapp, Rebecca Sheridan, Steven Oliver, Laura Jefferson, Karl Atkin
University of York
Background: The 2-week-wait urgent referral policy in the UK has sought to improve cancer outcomes by accelerating diagnosis and treatment(1). The policy also had potential to reduce social inequalities and geographical variation in outcomes.(2) More than 1.9 million 2WW referrals are made annually.(3) Almost half of all cancers are identified through this route, though for 92% of patients, referral will exclude cancer.(4)
However, around 5–7% of symptomatic referred patients cancel or do not attend their hospital appointment.
Study aims were to:
identify patient-level and practice-level predictors of non-attendance, and analyse outcomes including cancer diagnosis and mortality rates, and
examine how interpersonal, communication, social, and organisational factors influence a patient’s non-attendance.
Methods: Mixed methods study: a cohort study of data from a single NHS hospital over 7 years, and a qualitative study of interviews with patients who had cancelled or not attended their appointment, and GPs.
The study sample comprised all adults currently registered with a general practice within the three Leeds CCGs, referred to Leed Teaching Hospital Trust from April 2009-December 2016 on the ‘urgent referral pathway for suspected cancer’ (2WW) pathway. 'Non-attendance' was appointment cancellation or non-attendance.
Associations between non-attendance and individual and ‘practice-level’ factors were explored using multilevel logistic regression. We assessed rates of cancer diagnosis within six months of index referral date and, in patients diagnosed with cancer, any-cause mortality within 12 months of diagnosis.
We undertook individual interviews with GPs from practices that had agreed to participate, and wrote to non-attending patients from those practices. Interviews were semi-structured, informed by a topic guide, and data were analysed using the Framework approach to derive themes.
Results: 109,433 patients registered at 105 general practices in Leeds were included. A total of 5673 (5.2%) patients did not attend their index 2WW referral (non-attended).
Non-attendance was largely predicted by patient factors (age, gender, deprivation, suspected cancer site, earlier year of referral, distance to hospital) over practice factors. 10,360 (9.6%) patients were diagnosed with cancer within six months of referral (9.8% attending patients, 5.6% non-attending patients). Among these patients, 2029 (19.6%) died within 12 months of diagnosis: early mortality risk was 31.3% in non-attenders and 19.2% in attending patients. Data on cancer stage at diagnosis were available for a small proportion of patients (1,693, 16.3%). Patients who were not seen at 2WW had higher rates of stage 4 cancer (34.6%) compared to those who were seen (18.4%), as did patients who died within one year of cancer diagnosis (62.0% vs 12.2%).
We interviewed 21 GPs and 24 patients who had not attended.
There were some relatively straightforward explanations for non-attendance. However, other reasons were complex and related to patient expectations and social context, in addition to communication within the consultation.
Implications: There are implications for primary care and cancer services.
Effective interventions are needed to ensure greater proportions of patients present to the GP with earlier stage disease and attend referral appointments.
The effect reported here may be mediated through multi-morbidity, lower health literacy, or deprivation.
Patient responses, and especially the provoked worry, influence decision making, and occur within a social context and need to be negotiated by referring GPs. The urgent referral process should accommodate patient circumstances, perceptions, and responses, while ensuring an appropriate infrastructure in general practice and cancer services to facilitate referral, patient attendance, and responses to non-attendance.
NHS England. The NHS cancer plan: a plan for investment, a plan for reform. 2000.
Rachet B, et al. Br J Cancer 2010;103(4):446–453.
Samuels M, et al. Waiting times for suspected and diagnosed cancer patients: 2016–17 annual report. 2017.
PHE. Cancer services: demographics, screening and diagnostics. 2019.
Forecasting demand in secondary care: short-term non-elective admission forecasting
Emily Eyles(1), Tim Jones(1), Tim Keen(2), Marion Prat(3), Maria Theresa Redaniel(1)
(1)NIHR ARC West, (2)North Bristol NHS Trust, (3)University of Bristol
Background: The management of demand is a challenge faced by many hospitals. Non-elective (NE) admissions have become an urgent issue for many hospital administrations, putting pressure on already limited resources. For example, unplanned surgeries may cause planned surgeries to be rescheduled. Optimal patient flow management can be achieved through forecasting levels of demand in advance, on varying horizons, to give health-care staff an opportunity to prepare and allocate resources. The North Bristol Trust (NBT) has seen an increase in NE admissions in recent years, exerting pressure on staff and resources. Forecasting short- (days to weeks) and long-term (up to a year) demand is an essential component of operations planning at NBT.
The current models look at the number of attendances for the 13 weeks prior, adding in the percentage share that that same period had in the previous year as well as a 3% growth factor. The advantage of this model is that it requires little data from outside sources, and simplicity. These models used by NBT have been inadequate in forecasting admissions, due to difficulties with precision and accuracy, having an adverse impact on services, staff, and finances. More accurate methods of forecasting are warranted. This project aims to use econometric forecasting models in the context of the NBT, specifically for the daily short-term forecast, over several horizons.
Method: We used non-elective admission data provided by NBT covering the period between April 2015 and October 2019. This comprises 142,459 patient admissions over 1,675 days, with an average of 85 patients per day. The dataset includes demographic characteristics of patients, as well as diagnosis and procedure codes and length of stay.
Lagged temperature and precipitation were used in the model. Open access data on weather conditions (precipitation and temperature) were obtained from a nearby weather station. A deterministic trend variable (i.e. which direction admissions are trending), a weekday variable, and a ‘holiday’ variable (bank holidays, Christmas, and Easter, obtained from Government websites) were created and included in the model.
The daily forecasting models were created by splitting the data into training and test datasets. The training data comprised 1585 days, and the test data 90 days. AutoRegressive Integrated Moving Average (ARIMA) models were employed, as they examine past values and use them to effectively forecast. R’s auto.arima function was used to test for the best ARIMA specification on the training data. The ARIMA model was used to generate the forecast on the test data, along with 95% confidence intervals.
Results: The model is accurate within 10 beds over a three-week horizon, and then accurate within 25 beds within a six-week horizon. We will be comparing these forecasts with those currently produced by NBT. We will also be forecasting by specialty: general admission and surgery. We will test other forecasting methods.
Implications: If more effective than current methods, our proposed forecasting algorithm will enable NBT to plan resources more effectively and efficiently. We anticipate that our forecasting models will enable NBT to strategically long-term plan resources and enable them to mitigate surges in demand. The models can be used to forecast different admission types, or for different departments in the hospital.
Other Trusts or hospitals can use our method to generate their own forecasts. Weather data are freely obtainable, the deterministic trend can be generated from the training data, holiday dates from the Government, and the admission data is available from the Trust itself, or from HES. This means that because of ease of data access and code sharing arrangements, this forecasting strategy, which is also more precise, is easily reproducible in other contexts.