Mammography CAD recalls baffle CADET II researchers

Mar 19, 2009

Why do radiologists reviewing mammograms accept some prompts made by computer-aided detection (CAD) software, while others are rejected? Even researchers from one of Europe's most prominent CAD studies aren't sure, according to research presented at this month's European Congress of Radiology (ECR) in Vienna.

In an ECR session on breast screening and diagnosis, Dr. Fiona Gilbert from the University of Aberdeen in Scotland presented results from the Computer-Aided Detection Evaluation Trial II (CADET II) study, which compared the effectiveness of single readers using CAD to that of the gold standard in breast screening, two double readers.

Results from the main CADET II study were published in October 2008 in the New England Journal of Medicine, with Gilbert and colleagues finding that a single reader using CAD had comparable sensitivity and specificity to that of two double readers working without CAD. But the single reader+CAD combination had a higher recall rate, at 3.9%, compared to 3.4% for double reading.

With their more recent analysis, Gilbert and co-researchers sought to better understand the recall decisions made by the single reader using CAD, and how they differed from the double readers. The group analyzed mammograms from 28,204 women collected at three institutions in the U.K., and compared recall decisions between the two groups of readers, analyzing what types of cases were recalled and with what frequency.

The group found 227 cancers in the study population, with CAD marking the cancer in 80% of cases. The single reader using CAD and the double readers were in total agreement in ordering recalls on 170 cases. Of the 57 discordant cases, recalls were issued only by the double readers in 29 cases, and only by the single reader with CAD in 28 cases.

In analyzing CAD prompts by lesion type, the CAD algorithm issued correct prompts in 87% of soft-tissue masses and 100% of calcifications. The group of cases in which the two reading groups reported discordant recalls had a slightly higher percentage of soft-tissue masses. The percentage of microcalcifications recalled was very similar between the groups. The researchers also found little difference between the discordant cases in terms of lesion size and subtlety.

Of the discordant cases, 84% of the cases with the correct CAD prompt were recalled only by the single reader using CAD -- the double readers didn't see them. But in cases in which the double readers recalled cases that the single reader+CAD didn't, 68% had been correctly prompted by CAD -- meaning the single reader was ignoring correct CAD prompts.

On the positive side, the single reader using CAD still recalled patients with lesions that weren't prompted by the software. "This is in some ways quite reassuring because there are a number of false prompts that can distract readers," Gilbert said. "However, the single reader was ignoring 12% of cases that had correct prompts."

Gilbert concluded by stating that the CADET II data indicate that CAD has an impact on single readers interpreting mammograms. But how that process works is anyone's guess.

"When we tried to find out if there were obvious differences between those cases where the double reader only recalled the case and those only recalled by the single reader with CAD, there were no obvious differences in terms of mammographic features, size, or visibility of the lesion," Gilbert said. Collaboration is needed with other CAD researchers to better understand why readers are ignoring correct prompts, she noted.

In a related presentation, Dr. Matthew Wallis of Addenbrooke's Hospital in Cambridge offered an overview of CADET II data with respect to CAD's impact on different types of readers. In the U.K., specially trained radiographers are frequently used to view films in breast screening programs due to a shortage of breast radiologists. The researchers wanted to determine if CAD was used differently by either group, as well as by breast clinicians, a third type of physician trained to bring a multispecialty approach to breast care.

According to the data, radiographers had higher recall rates than radiologists (5.1% versus 3.4%), with breast clinicians experiencing a similar recall rate to radiologists (3.7%). The group also found that less experienced readers using CAD alone had higher recall rates than the double readers. Less experienced readers also had higher cancer detection rates when working as a single reader with CAD as opposed to a double-reading environment, though the difference was not statistically significant.

"Single reading with CAD is a viable alternative to double reading," Wallis said. "For the least experienced, single reading with CAD has the highest cancer detection rates, but at a cost of a higher recall rate."

By Brian Casey
AuntMinnie.com staff writer
March 20, 2009

Breast MRI CAD algorithm predicts cancer metastasis, February 6, 2009

CAD can boost double-reading detection rate, January 22, 2009

Interactive use of mammo CAD may improve mass detection, December 22, 2008

Breast MRI pinpoints residual disease after chemotherapy, December 19, 2008