A study evaluating AI-based applications for intracranial hemorrhage (ICH) shows they have high diagnostic accuracy, particularly in assisting radiologists, but information about their therapeutic impact and clinical utility is still limited, according to Dutch healthcare AI specialists.
"The broader impact on workflow and therapeutic outcomes remains unclear," noted Stephan Romeijn, co-founder of Romion Health in Utrecht, and colleagues. "Studies indicate potential benefits, but the lack of independent, higher-level efficacy research limits conclusions on their clinical utility, highlighting the need for robust validation and real-world clinical impact assessments."
The authors assessed all CE-marked AI-enabled applications for ICH management. Through HealthAIregister.com, of which Romeijn is co-owner with Kicky van Leeuwen, PhD, the researchers identified 11 applications: Annalise Enterprise CTB, BrainScan CT, CINA-ICH, DeepCT, Deepstroke, Intracranial Hemorrhage (ICH), JBS-04K, qER, Rapid ICH, StrokeViewer, and Viz ICH.
They collected peer-reviewed papers on the software from the product pages on both the Health AI Register and through a PubMed search. They then extracted the study design, study population demographics, outcome measures, and any vendor affiliations, categorizing the outcome measures using a hierarchical model of efficacy of diagnostic imaging.
However, of the 19 publications the researchers reviewed, 11 were related to a single AI product; they were unable to identify publications related to two products. Only two publications used European data, they reported in a poster presentation at ECR 2025.
Image courtesy of Stephan Romeijn et al and presented at ECR 2025.
For the researchers’ hierarchical model of efficacy (designed with levels up to 6), 15 studies were designed to address the applications’ diagnostic accuracy efficacy (level 2). The higher levels of the model, in which the impact on clinical workflow was examined, were addressed in nine studies, with five focusing on therapeutic efficacy (level 4) and four on diagnostic thinking efficacy (level 3).
In the literature, the AI applications demonstrated high sensitivity, ranging from 87.2% to 97.84%, as well as high specificity (from 84.4% to 99.52%).
Two studies on the qER and the Rapid ICH applications examined the volumetric quantification function, reporting an intraclass correlation coefficient (ICC) of 0.96 [CI: 0.88-0.98] for qER and a correlation coefficient r of 0.983 for Rapid ICH, Romeijn and colleagues pointed out.
The studies examining the clinical impact of the applications generally yielded less definitive and more mixed results, with several labeled as “has potential” and either “mildly convinced” or “still inconclusive” in the researchers’ analysis.
While the authors said the applications in their review yielded high rates of accuracy across studies for both sensitivity and specificity, data on workflow impact and clinical utility were still lacking, especially higher-level data.
"It's clear that there's more beneath the surface when examining these publications, revealing both biases and areas for further investigation," stated co-author Jimmy Banken in a blog on this topic.
He added that the group's exploration into AI products aiding in ICH management showed the following:
- The potential for AI products to revolutionize clinical practice in ICH management is promising but not yet conclusive on all the clinical impact factors.
- Issues with proving higher-level efficacy and the lack of comprehensive EU data remain significant hurdles.
- Many products lack the robust, independent research validation that their vendors often claim.
"As we continue to advance in the field of AI in healthcare, it is essential to approach these products with a critical eye, ensuring that they are not only innovative but meet their promise when adopted in clinical practice. Only then can we fully harness their potential to make our healthcare system futureproof," he noted (see healthairegister.com/news/2024/09/03/impact-of-ai-in-intracranial-hemorrhage-management-current-products-and-evidence/).
You can read the full ECR poster here.
