New clinical evidence has shown the potential of artificial intelligence (AI) to detect cancers missed by human readers and boost detection rates without recalling extra false positives, according to a European research team.
Current AI systems developed for detection of breast cancer have reported stand-alone performance comparable to humans, noted first author Dr. Jonathan Nash, medical director for U.K. and Ireland at Kheiron Medical, and his colleagues. Several retrospective studies have demonstrated the ability of AI to detect some of these false-negative cancers, but there is no prospective data to quantify AI's clinical utility, they added.
Against this background, the researchers' aim was to demonstrate real-world postmarket benefit of AI as an extra reader in breast cancer screening. They presented their findings in a poster at the annual scientific meeting the Royal Australian and New Zealand College of Radiologists (RANZCR), held recently in Adelaide.
Study logistics
The team implemented a commercially available AI system at a large breast cancer screening facility in Hungary to analyze images of women who participated in the national breast screening program or who came to the clinic for opportunistic screening within a six-month period in 2021. The AI system was employed as an extra reader, in addition to standard human double reading, and its recommendation was based on predefined operating points.
AI was used to flag cases for additional review to support services to not miss cancers. In this way, AI served as a safety net to double reading and to support early detection of cancers in women who would otherwise be asked to come back at their next screening interval.
The group's extra reader (XR) workflow uses standard double reading, complemented by an assessment of the images by AI. If double reading results in "no recall" while the AI flags a case, the screen is assessed by extra human arbitration. Figures courtesy of Dr. Jonathan Nash et al, presented at RANZCR ASM 2022.The extra reader workflow involved flagging cases the AI suggested to recall which human double-reading did not recall (positive discordant cases). Subsequently, these cases were additionally arbitrated by a radiologist, and all detected cancers were confirmed by pathology.
Over six months, 3,746 patients were screened. Of the cases that were not recalled by human double reading, the AI flagged 396 cases for extra review by a human reader, amounting to a positive discordance rate of 10.6%, Nash and colleagues reported.
Human double reading had a recall rate of 6.7%, an arbitration rate of 3%, and a cancer detection rate of 12.5/1,000 (47 cancer cases out of 3,746 screens).
Extra arbitration of the screens flagged by the extra reader (XR) resulted in recalling six women who were not recalled by standard double reading. All six women were subsequently diagnosed with breast cancer. (HDR = human double-reading)Extra human arbitration, based on the use of an extra reader, resulted in recalling six patients, all of whom were diagnosed with breast cancer. This equated to a total arbitration rate of 13.6% (510/3,746), and an absolute increase in cancer detection rate of 1.6/1,000 (total cancer detection rate of 14.1/1,000), they pointed out.
An exact simulation with a less sensitive AI operating point yielded an extra arbitration rate of 2% (89/3,746), while still detecting five of the six cancers.
Benefits of AI
The importance of breast screening programs is evident, the authors wrote. Mammography has its limitations and a significant number of cancers are not detected in the regular screening round but are diagnosed as interval cancers during follow-up; of these, approximately 13.6% to 35% are retrospectively classified as having been missed (false negative) in screening, they continued.
Overall, the prospective real-world data gathered from the service evaluation at a single site provides evidence for the potential of AI to detect cancers missed by human readers and increase cancer detection rates without recalling extra false positives, the researchers concluded.
"The results provide prospective real-world evidence showing the benefit of using AI as an extra reader in breast cancer screening," they noted. "Combining the extra-reader workflow with workflows focused on workload savings will mitigate the increased arbitration rate and optimise clinical and operational benefits."
Dr. Éva Ambrózay, a radiologist at the MaMMa Screening and Diagnostic Center in Budapest and board member of the breast section of the Hungarian Society of Radiologists, was a co-author of the RANZCR poster. The other co-authors were Dr. Peter Kecskemethy, CEO and co-founder, and Dr. Edith Karpati, medical product director, both of Kheiron Medical; Prof. Ben Glocker, professor in machine learning for imaging at Imperial College London; and Cary Oberije, PhD, Annie Ng, PhD, and Georgia Fox.
