Evidence from users based in 145 countries suggests an online diagnostic training tool is showing considerable initial promise as a way of improving the capacity to detect early-stage COVID-19 on chest CT. The Australian-led project has identified important regional discrepancies, however.
The cloud-based CovED training module, developed by the University of Sydney spin-off company DetectED-X and launched in April, is available free to all physicians at detectedx.com. By using the tool, radiologists and other doctors are able to more quickly and accurately assess COVID-19 cases on lung CT, accessing the case-based training within seconds, DetectED-X chief executive officer Prof. Patrick Brennan told listeners at a GE Healthcare webinar on 16 June.
The training module comprises 30 clinical images drawn from the global image bank. Participants are asked to describe each image in turn by clicking on various description boxes, such as ground-glass opacities, crazy paving, and consolidation. They are also asked to define the distribution by choosing between upper or lower zone, left or right, bilateral, posterior, anterior, central, or peripheral locations. The technology has typical tools, including the ability to zoom in, pan, or alter contrast and brightness, and all images are visualized at full resolution.
The module includes real COVID-19 CT images at full resolution. Here the interface reveals a true COVID-19 case. All images courtesy of Prof. Patrick Brennan."In each case, the radiologist must make a judgment on the image. The last thing is to say how confident you are that this is a COVID-19 infection on a scale from 0 to 5, and from there the system scores sensitivity and specificity and other categories such as false positives," Brennan explained.
The module can take from 15 to 45 minutes, depending on how fast the radiologist completes each question. The tool is also accredited, and by completing the training, participants can gain an automatically generated certificate of attendance with associated continuing professional development (CPD) and continuing medical education (CME) points.
Participants' sensitivity and specificity scores, as well as any false positives and negatives, are produced at the end of the session, when the true COVID-19 cases are revealed. The system also allows the participant to compare their findings per image with that of the expert group to see if they were in agreement with the consensus. In addition, doctors can choose to anonymously rank themselves.
"Essentially the tool consists of three parts: You look at the image and make a judgment, it tells you how good you are, and it allows you to judge your opinions against the opinions of an expert group. So within a short period of time, you can have a ... sense of how good you are and if you think you need improvement in certain areas, there are more test sets on the website that you can use to enhance your abilities," Brennan said.
Key results so far
The tool can bolster significantly the capacity of doctors to spot early COVID-19 cases on chest CTs, but data generated since April show that there are still variations between individuals, with an average variation on the AZ graph of between .82 and .92 at one given time point, he continued. Regional variations between Europe, Asia, Australia and New Zealand, the U.S., and the Middle East also exist.
"We expect the variation to be there, but we hope our tool will minimize that variation," he noted.
Table shows receiver operating characteristic (ROC) values for training-generated data across different reader types at one given time point.
Table shows regional ROC values from data generated by training participants at one given time point.Brennan applauded the industry partners, including GE Healthcare, Volpara, Amazon, and the World Continuing Education Alliance (WCEA), for helping to distribute the tool that could make a huge difference to how quickly doctors are able to identify and treat the disease during the pandemic. Notably, the technology could drastically improve patient management in developing countries.
"It was one thing developing the tool, but the critical thing that the companies were good at was making sure that it was available to clinicians everywhere," Brennan noted. "The servers are robust and secure and it is available to anyone, anywhere. We want as many people to use it as possible."
Potential applications
The CovED tool originally sprang from BREAST, the University of Sydney and DetectEd-X's breast cancer diagnosis training platform, which is used by around 90% of doctors in Australia and New Zealand. Data generated from the training platform shows proven improvement in diagnostic skills and a reduction in diagnostic performance variation over time, according to Brennan. CovED was developed in response to the World Health Organization's call for solidarity in the global response to COVID-19, but Brennan now sees wider uses for the technology.
"There is a need for this tool for all parts of the body, and I don't see why it can't be used for liver lesions, brain hemorrhages, or fractures. If there are image types you want to see used, we're happy to try that," he noted.
Brennan called for more collaboration and invited members of the audience to use the tool and send him their feedback, and even offer their own suggestions for extending the tool to other types of systems and body parts. U.S. colleagues from the University of Missouri, for example, have suggested a module for training in liver lesion diagnosis.
"We can use any images, 2D and 3D, and look at all slices for any part of the body," Brennan noted, adding that the developers plan to create a platform with its own app suitable for smaller devices such as iPhones and Android devices.
A second webinar speaker, Prof. Stuart Grieve, chair of radiology at Sydney University, noted that when the pandemic settles into a more steady phase, CT may develop a more significant role. He stated that CT may become increasingly relevant for screening in the context of trials associated with COVID-19.
Yet to be stored in the iBank are long-term disease images, due to the relatively recent emergence of the disease, Grieve noted. "That's a work in progress. We're trying to understand exactly how much residual disease individuals with moderate or severe infection are going to be left with."
