'Priming' doesn't work for mammograms, study finds

When it comes to breast cancer screening, inserting additional malignant cases into datasets of screening mammograms may lead to a decrease in the performance of experienced breast radiologists, according to a new study in the British Journal of Radiology (BJR).

The effectiveness of screening mammography relies heavily on reader performance, and the challenge for radiologists is to gain experience catching abnormal mammograms while still keeping their recall rate down.

It's been proposed that inserting "test malignant cases" into datasets of "real life" clinical screening mammography cases may increase cancer detection rates by artificially increasing the prevalence cases, according to lead author Sarah J. Lewis, PhD, and colleagues (BJR, July 2014, Vol. 87:1039). However, by increasing the prevalence of abnormalities, there may be an inadvertent effect on the detection of subsequent "real" cases.

"Previous work on the impact on increasing abnormality prevalence on radiologic performance has provided diverse conclusions, with one study suggesting that varying prevalence was unlikely to alter the accuracy of the radiologists, another suggesting increased diagnostic efficacy with increasing prevalence, and a further arguing no significant effect," wrote the authors from the University of Sydney in Australia.

Lewis and colleagues drew on cognitive psychology for their research: Human visual perception of elements in the environment is greatly dependent on what was previously seen, so, for instance, detecting a target beforehand helps in picking it out of the environment. This is called "perceptual priming" and is an underexplored area of research in radiology.

"In this context, priming can be understood as a mechanism that 'highlights' the presence of a given target by increasing the saliency of the stimulus," Lewis and colleagues wrote. "It is generally accepted in the cognitive psychology literature that priming can be used to direct visual attention toward an explicit target, and that observers become proficient at finding targets when salient prompts are employed."

The phenomenon has real relevance in clinical imaging, not just to improve diagnostic accuracy, but also to establish the effects of audit cases randomly inserted into a reading set to test radiologic performance, according to the researchers. Therefore, the group sought to measure the effect of adding malignant cases of lower difficulty, which they called primers.

Two studies were crafted to test their hypothesis. In the first study, eight radiologists read two sets of images each consisting of 40 mammography cases. Set A contained four abnormal cases, and set B contained six abnormal cases, including two priming cases placed at intervals of three and five subsequent cases before a subtle cancer.

In the second study, 16 radiologists read a third set of the same cases, known as set C, containing six abnormal cases and two priming cases immediately preceding the subtle cancer cases. The readers were asked to localize malignancies and give confidence ratings on decisions.

The researchers found a decrease in performance in set B compared with set A, although it wasn't significant. However, there was a significant increase in the receiver operator characteristics (ROC) area under the curve (z 522.532; p 50.0114), and location sensitivity (z 522.128; p 50.0333) between the first and second halves of set A. There was a marginal improvement in jack-knife free-response ROC figure of merit (JAFROC FOM) (z 521.89; p 50.0587) between the first and second halves of set B.

In the second study, set C showed no significant differences between the two halves of the study.

"For set A, there was a statistically significant increase in the performance of the radiologists in the second half of the test set, where we observed an increase in location sensitivity without a change in specificity," the researchers wrote. "One possible explanation for this increase may be that the radiologists were becoming familiar with the experimental setting and were developing a sense of how likely it was that they would find cancers within the test set."

For set B, where the primer cases were present, no such improvements were noticed, although the JAFROC FOM increased from the first half (mean 50.570) to the second half (mean 50.829), which was marginally significant. This was accompanied by a small nonsignificant improvement in specificity (first half mean 50.559; second half mean 50.640).

"Taken together, these results suggest that the primers may have had some effect on the breast radiologists," Lewis and colleagues wrote. "Unfortunately, we cannot assess which effect this may have been, because of the potential impact of memory effect, which cannot be disregarded in this experimental setting."

It is also possible that memory traces from the first reading influenced the second reading, especially owing to the short time lapse between the readings, the researchers noted.

"This work was an exploratory study to determine if any perceptual priming could be observed by the insertion of relatively easy positive cases," they wrote. "Whilst no direct evidence was shown for priming, some small changes found here need further study."

Future studies should address the timing of the cases and the practical difficulty in anticipating when abnormal cases occur, technical challenges with inserting cases seamlessly into a radiologist's workload, and the type of lesion that should be used as a primer.

"All these are currently being considered by our group and future work will focus on these issues, the removal of any memory effect, and the impact of priming on less experienced radiologists. The number of both normal and abnormal cases included within each test set will be increased," they concluded.