Use of artificial intelligence to identify patients to be assessed in a breast clinic on 2-week wait: a retrospective cohort study (2024)

Associated Data

Data Availability Statement

Abstract

Key words: artificial intelligence, breast cancer, breast surgery

Introduction

Breast cancer remains the most common cancer in females in the UK, with 55900 new cases diagnosed annually¹. Patients with breast symptoms frequently present to their general practitioner (GP)². The ‘two-week wait’ initiative was launched to provide guidance for GPs to triage referrals to specialist breast clinics as urgent or routine, to ensure those with suspected breast cancer or ‘red flag’ symptoms are prioritised to be seen within 2 weeks³. Since its introduction, the number of primary care referrals to one stop breast clinic has continued to rise beyond the capacity of the 2-week wait pathway^4,5. Despite the increased number of referrals, the proportion of women referred urgently who are identified as having breast cancer is decreasing⁴. The 2-week wait criteria for suspected breast cancer has subsequently been criticised as having a poor predictive value despite revisions to address this^3,6. Guidelines were modified in 2015 to lower the threshold for urgent referrals from 5 to 3% risk of breast cancer. However, this had no significant impact on sensitivity and a lower specificity^3,6. Furthermore, studies have shown a significant number of referrals to one stop breast clinic are inappropriate according to NICE guidelines, with an increasing number of urgent referrals for patients with benign breast conditions or absent pathology, who do not present with ‘red flag’ symptoms^4,7,8.

To improve adherence to referral guidelines, proformas have been widely implemented, although inconsistent or incomplete use has limited their impact⁹. Consultant and nurse-led telephone triage have also been trialled, requiring significant additional resources, and failing to address the issue of rising referral numbers^10,11. Across healthcare, self-referral pathways have been efficaciously used with high levels of patient and clinician satisfaction, reducing appointment loads for GPs and reducing delays for specialist assessment¹². We propose the use of a self-referral pathway using an online algorithm-based application to triage patients with breast symptoms according to NICE guidelines, to be seen in the breast clinics as urgent, routine or by their GP. The sensitivity and specificity of the online software for identifying patients who required urgent review were 100% and 98%, respectively⁸. This bypasses the role of GPs as gatekeepers, reducing the demand on primary care services and avoiding unnecessary delays arranging GP currently required for referral to specialist services whilst ensuring referral guidelines are adhered to.

A significant benefit of using online triage software in this way is its ability to acquire large quantities of data. Artificial intelligence (AI) is being increasingly used in medicine to analyse big data to identify predictive patterns and complex associations to solve clinical problems. AI has been successfully applied in medicine to support clinicians with diagnosis, treatment, and prognostication of outcomes across a variety of specialities and its use has been endorsed in the NHS long-term plan^13–15. Machine learning artificial neural networks (ANN), like the human brain, learn from experience and can analyse vast quantities of nonlinear, imprecise data to accurately recognise patterns. ANNs have been used extensively in the analysis of radiological images and histological specimens^13,14. In breast surgery, they have been used to detect nodal metastases from breast cancer and interpret mammograms, acting as a ‘second reviewer’ in breast cancer screening, in some cases outperforming the expert reviewer^16–19. Additional applications have included the prediction of breast cancer progression and survival with a high degree of accuracy^20,21.

Crucially, AI has been used to triage referrals to secondary care. Moor et al.²² have used this technology to accurately classify patients with ‘predicted cancer’, referred under the 2-week wait pathway for suspected head and neck cancer. NHS Lothian has implemented an AI triage system for referrals to the gastroenterology department, which is able to make predictions on urgency to direct patients to various treatment pathways²³. In our study, we used machine learning ANNs to identify patients with the likelihood of having a biopsy as a surrogate marker for those who need to be seen urgently in a specialist breast clinic. Integration of AI technology into self-referral applications will enable the software to learn from the data, to effectively triage patients with breast symptoms in addition to bypassing the need to see a GP.

Methods

This study has been registered with the research Registry. Unique identifier number: researchregistry9078. Available at https://www.researchregistry.com/browse-the-registry#home/registrationdetails/646fd8910743f20026e237a6/. The patients’ involvement in the study was ascertained in an earlier study where patient’s perceptions of self-referral questionnaire to breast clinic was obtained¹². Majority of the patients opinionated for a self-referral pathway to breast clinic as compared to GP referral network. Data was collected retrospectively from the electronic medical records of new symptomatic patients attending one stop breast clinic at Broomfield hospital, Chelmsford (Mid and South Essex NHS Trust). Data were collected for patients attending breast clinic from 1 June to 1 October 2021 to assess patient characteristics and predictive factors associated with having a biopsy. Patients with new breast symptoms who attend the one stop clinic are required to complete a questionnaire about their presenting complaint, relevant past medical and surgical history, drug history, and social history. The second part of the questionnaire form is completed by the surgeon who examines the patient. This consists of examination and radiological findings, details of the core biopsy and clinical impression after the patient’s assessment. The information from the questionnaire was uploaded into the electronic health record system of the patient. Data was collected on the patient responses to the questionnaire, clinical findings and outcomes and transferred to Microsoft excel. The total dataset was 794 patients with 66 patients having had core biopsy in the clinic. For machine learning of AI, the data was split into two: training and testing datasets. There were 460 patients in the training dataset to train AI and the other test dataset included 334 patients. The data was split randomly by the machine learning, but more patients were programmed to be in the training dataset so that we can improve the accuracy of the machine learning AI model.

The artificial intelligence application was built on PHP using a machine learning library. Three commonly used models were used to identify patients who had the core biopsy of the breast lesion. These models were K-means clustering, K-neighbours algorithm, and neural network (multilayer perceptron classifier). The neural network mimics human brain functionality by recognising the underlying relationships in a dataset. The CSV file dataset was used for machine learning and testing application. The dataset was applied to the app with already incorporating clinical variables that were collected from the patient questionnaire to predict the likelihood of having a biopsy in the clinic, which acted as a surrogate marker of identifying those patients which would be prioritised to be assessed in the clinic. The data was structured with known outcomes. The input variables into the model were based on past medical history and risk factors for breast cancer as assessed by the questionnaire given to the patient routinely when they visit a one stop breast clinic. There were 11 dichotomous variables as follows: sex, personal history of breast cancer, previous removal of benign breast lumps, previous breast implant or reduction, previous mammogram, history of hysterectomy, history of oophorectomy, use of hormone replacement therapy (HRT), family history of breast or ovarian cancer, history of diabetes, and history of obesity. There was one categorical input variable, the presenting symptom (breast lump, breast pain, change in shape/skin problem, lumps felt by GP, incidental findings on a scan, nipple discharge). There were 4 continuous variables as follows: age, duration of symptoms in weeks, the number of years the patient used HRT, and number of relatives with breast or ovarian cancer. The data were randomly split into two; 460 patients from the data were used for machine learning of the application and 334 patients were used to test the application. Each group also had similar patients who had core biopsy (32 in test data and 34 in training data). This was more important as the primary outcome of the AI app is to identify these patients. The coding of the software was performed by Mr Hassan Ali, who is an IT software engineer at Gnovatech Pvt Ltd, UK.

The work has been reported in line with the STROCSS criteria²⁴. The ethical approval of the project was obtained from Broomfield Hospital department of audit and service improvement, Mid and South Essex NHS Trust. The local study ID was CA21-104. Descriptive analysis was performed using Microsoft Excel. As it was a pilot study, the number of participants included in the study were based on the total number of referrals seen in the breast clinic in 6 months. No power calculation was carried out. The comparison of the dichotomous and continuous variables was conducted using χ² and independent t-test, respectively. A P value of less than 0.05 was considered significant.

Results

A total of 794 cases were referred to one stop breast clinic for new breast symptoms—37 male (4.6%) and 757 female (95.3%). The average age of the patients included was 43.2 years. Five hundred thirty-six patients (67.5%) presented with a breast lump, 180 (22.7%) with breast pain, 61 (7.7%) with changes to shape or skin and 13 (1.6%) with a lump identified by their GP. Five hundred twenty of the cases identified presented with more than one symptom (65.4%). The most common second symptom was breast pain (n=275, 34.6%), lump felt by the GP (n=154, 19.4%) and change in shape or skin (n=89, 11.2%). The average duration of symptoms was 12.8 weeks. Twenty-four (3.0%) patients had personal history of breast cancer. Forty-eight patients (6.0%) had a previous history of breast implants. There were 66 patients (8.3%) who had used HRT and the average duration of their use was 0.6 years. There were 285 patients (35.9%) with a family history of breast or ovarian cancer, 163 patients had first-degree relatives affected with breast or ovarian cancer (20.5%). The following number of patients had other relevant co-morbidities: diabetes (n=11, 1.2%) and obesity (n=19, 2.1%). There were 233 patients (28.2%) that did not meet the NICE guidelines for urgent referral to a 2-week wait one stop clinic.

Table ​Table11 shows the common characteristics of the patients who had a biopsy compared with the patients who did not have a biopsy. There was a greater proportion of patients who had a biopsy that were older [52.8 (SD 17.9) vs. 44.1 (SD 16.8), P<0.001], had a previous mammogram [n=21, (31.8%) vs. n=148 (20.3%), P 0.03], benign breast disease[n=9 (13.6%) vs. n=23 (3.1%), P<0.001], hysterectomy [n=7 (10.6%) vs. n=39 (5.3%), P 0.03], and HRT [n=13 (19.7%) vs. n=53 (6.4%), P<0.001] compared to those who did not have a biopsy. Most of the patients who had a biopsy had either breast lump (n=59, 88.0%) or shape/skin changes on the breast (n=6, 8.9%). Only three patients (4.4%) had breast pain. These patients also had a significantly higher proportions of presentations with a breast lump (P<0.001) and a significantly lower proportions of presentations with breast pain (P<0.001). These patients with breast pain having had a biopsy had a benign (U2 or M2) imaging.

The sensitivity to identify patients who had core biopsy for K-means clustering, K-neighbours algorithm, and neural network algorithm were 72%, 70%, and 84%, respectively. Since the sensitivity of neural network was higher than other, we also calculated its specificity which was 90%. The positive predictive value (PPV) was 47.4% and the negative predictive value (NPV) was 98.2%. The ratio of PPV to NPV was 0.48.

Discussion

In our study, the most common presenting symptom of patients attending one stop breast clinic was a breast lump followed by breast pain. A large proportion of patients who were referred via the 2-week wait pathway did not meet NICE guidelines for urgent referral. Increased age, use of hormone replacement therapy, previous mammogram and benign breast disease were factors significantly associated with patients having a biopsy. The AI application using neural networks was most accurate in identifying those patients who are likely to have a biopsy in the clinic with a sensitivity and specificity of 84% and 90%, respectively. The AI application was used to identify those patients who had a biopsy in the clinic as a surrogate marker to detect patients that require urgent assessment by the specialist breast team, because they present with a suspicious finding that warrants a biopsy. This includes patients with cancer and those patients with benign lumps with suspicious features.

The positive predictive value of the AI app was 47.4% which is significantly The use of AI in the diagnosis and treatment of breast cancer has been assessed previously and is gaining a lot of research attention^16,17. AI has been used to accurately read mammograms and is being tested to provide a second review following initial assessment by the radiologist in the breast screening programme¹⁹. Similarly, the use of AI to predict multi-disciplinary team outcomes, management plans and prognosis is currently being evaluated²¹. The role of AI assessed in this study to identify factors requiring biopsy to streamline the patient care pathway and facilitate early diagnosis is novel.

Increasing referrals without a concomitant increase in the diagnosis of breast cancer is placing unmanageable pressure on specialist breast services, resulting in country wide breaches of the national 2-week wait target for suspected breast cancer assessment. COVID recovery has also placed additional demands on both primary and secondary care services. A variety of methods have been trialled to curb the number of referrals to the breast clinic¹⁰. These solutions have not gained interest widely due to the increased set up costs and staffing requirements.

Prediction tools based on patient history and risk factors have also failed to address the problem, as in this study, many of the patient risk factors and and characteristics were not significantly different for patients who ended up having a biopsy in the clinic. However, AI has the advantage of being able to identify complex associations and patterns from large data sets, which may provide us with a greater understanding of the multifaceted relationship of symptoms, signs and risk factors associated with breast cancer. This has the potential to be utilised to further refine referral guidelines as the AI software was able to effectively predict patients requiring biopsy in our study based on these factors, as recognition of predictive patterns may help identify those patients who have breast cancer but do not present with ‘red flag’ symptoms or have benign imaging¹⁶. This is where the use of AI may provide the most benefit, as the incidence of breast cancer in young women and those referred routinely without red flag symptoms is rising, and around 15–20% of breast cancer are mammographically occult^25,26.

The use of AI app provides an alternate pathway for patient referral compared to the current standard of ‘2 week wait’ pathway. The current pathway was introduced to increase early diagnosis of the breast cancer²⁷. However, the studies have shown that the PPV of the pathway is 4% and that’s also for patients with breast lump as a primary red flag symptoms²⁷. The PPV is even lower for patients with other breast symptoms and less 1% for breast pain. In comparison, the PPV for the AI app is 47.4% which is considerably higher than the standard referral pathway. The ratio of PPV to NPV was 0.48, which suggests that almost one in three patients referred through this system will have breast cancer, making it a very robust pathway.

There were certain limitations of the study that should be considered. It was a single centre study and patient characteristics may vary according to the region. Machine learning was also conducted on a retrospective dataset and therefore may not have included factors which may be pertinent to the study, such as ethnicity and smoking status. The number of patients with co-morbidities and other factors were also low in the study population. In the clinic, a generally healthy population is seen, although it may be possible that patients were unable to provide full details on the questionnaire, highlighting the issue of recall bias. A common criticism of AI applications is the requirement for large datasets to improve accuracy¹⁶. Although a relatively large dataset was used for machine learning and testing as used in other studies, the number of patients having a biopsy was small¹³. We assume that if there were more patients requiring biopsy, the machine learning would have improved.

The demand for one stop breast clinic assessment is increasing, and it is currently outstripping the national capacity of the service. Innovative methods of highlighting those at risk, and safely reassuring those who are not at risk, need to be developed. The current failure of the 2-week wait service serves neither those with cancer with delayed diagnosis nor those who are well but spend many weeks worried by being on a ‘cancer pathway’. The AI application is particularly helpful in identifying those patients with breast lumps who are at a higher risk of having a suspicious lump, and those patients who do not present with a breast lump but still have a risk of having a suspicious non-palpable lesion in the breast requiring a biopsy. By utilising machine learning on a larger data set, the accuracy of the AI can be increased to help identify these patients.

In this study, AI was able to accurately identify patients requiring biopsy from presenting symptoms, with a sensitivity and specificity of 84% and 90%, respectively. The need for biopsy can be used a surrogate marker for those requiring urgent assessment in one stop breast clinic and can be used to effectively triage patients with breast symptoms.

Ethical approval

The ethical approval of the project was obtained from Broomfield Hospital department of audit and service improvement, Mid and South Essex NHS Trust. The local study ID was CA21-104.

Consent

Not applicable.

Source of funding

The study did not receive any specific funding.

Author contribution

A.R. conceptualised the study, developed the patient questionnaire, collected data, analysed data, and edited manuscript. L.M. collected and analysed the data and wrote the manuscript. S.S. conceptualised the study, supervised the study and edited the manuscript.

Conflicts of interest disclosure

The coding of the AI software was done by Gnovatech Pvt Ltd. The company is owned by Mr Ghaus Rao, who is the brother of the corresponding author of the study, the authors declare that there is no conflict of interest regarding the publication of the study.

Research registration unique identifying number (UIN)

This study has been registered with the research Registry. Unique identifier number: researchregistry9078. Available at https://www.researchregistry.com/browse-the-registry#home/registrationdetails/646fd8910743f20026e237a6/.

Guarantor

The guarantor of the study is Mr Simon Smith.

Data availability

The data collected from the patients to be used to train AI are available on request to the corresponding author. The training and test data are also available on request to the corresponding author, A.R.

Provenance and peer review

Not commissioned, externally peer-reviewed.

Acknowledgements

The authors thank Gnovatech Pvt Ltd to help with the coding of the AI software to test the machine learning of the data.

Footnotes

References