Artificial intelligence (AI) algorithms can identify radiology reports that contain unexpected findings with a high level of accuracy, potentially enabling clinicians to be informed sooner about important results, Spanish researchers have reported.
A team led by Pilar López-úbeda of the University of Jaén trained a variety of deep-learning and machine-learning algorithms to find unexpected results in radiology reports. In testing, López-úbeda and colleagues found that a convolutional neural network (CNN) yielded 90% accuracy, slightly outperforming other deep-learning models as well as traditional machine-learning algorithms.
"The best developed system will be put into practice in the HT médica, where specialists will be able to evaluate radiology reports in common practice for the detection and notification of unexpected findings," the authors noted in an article published online on 18 June in Expert Systems with Applications.
Unexpected results
Although radiological findings are usually in accordance with the clinical suspicions or patient symptoms, it's not uncommon to discover unexpected results that aren't apparently related to the suspected diagnosis or reason for ordering the exam, according to the researchers.
"For example, the detection of pulmonary embolisms in an oncological patient that had undergone a routine follow-up [CT] exam for prostate cancer surveillance may be considered as an unexpected finding which must be immediately communicated to physician and patient," they wrote.
How AI can help
With the increasing trend for radiologists to communicate results to clinicians or directly to patients, machine learning can help radiology specialists make decisions when the algorithm detects a possible unexpected finding in the report, according to the authors.
The researchers sought to compare the performance of machine-learning and deep-learning algorithms for identifying unexpected radiology results. They trained a variety of machine-learning models, including naive Bayes, random forest, support vector machine, and logistic regression algorithms. In addition, they developed several deep-learning algorithms, including a CNN, a recurrent neural network, and a combined long short-term memory network and CNN.
To train the systems, the researchers used a Spanish anonymized corpus -- including annotated unexpected findings -- provided by HT médica in Jaén. Of the 5,947 reports provided by the institution, 2,922 were labeled as having unexpected findings and 3,025 did not have any unexpected results. They used 4,757 (80%) of the corpus for training and set aside the remaining 1,190 reports for the test set.
CNN performs the best
The CNN yielded the best performance, correctly labeling 1,072 (90%) of the 1,190 documents. The other algorithms also performed well, yielding accuracy ranging from 79.6% (naive Bayes) to 89.8% (linear regression).
There were 55 false-negatives and 63 false-positives. The researchers then delved further into the results to determine why these reports were incorrectly labeled by the algorithm.
One report, for example, that had been incorrectly classified as containing no unexpected findings included words implying normal results, such as leve (mild), pequeños (small), discreta (discreet), and bien (well). On the other hand, one false-positive report contained words such as: irregularidad (irregularity), aumentado (enlarged), tumoral (tumorous), hepáticos (hepatic), esplenomegalia (splenomegaly) and metastásica (metastatic).
The researchers are now considering an assessment of the impact of negation in their systems, as the correct treatment of negation clues such as "without" or "not" is paramount when working with clinical textual information.
"In addition, and since our system will be put into practice in the HT médica, we will enrich the corpus with new training samples because the clinic experts will continue to evaluate future radiology reports for early detection of unexpected findings," they stated. "Thus, the system will be retrained on new documents and it is expected that the accuracy will be increased."
