Researchers from Germany have developed artificial intelligence (AI) algorithms that can generate a 3D liver model from MRI or CT scans. They examined these 3D models to plan liver cancer surgery and also projected them onto the patient intraoperatively using augmented reality software.
The group, led by Dr. Andrea Schenk of the Fraunhofer Institute for Medical Image Computing (MEVIS) in Bremen, combined the various automated and semiautomated algorithms into computer software. By applying the software to MRI or CT scans, they were able to analyze, segment, and measure the liver tumors and vessels -- and, ultimately, create highly detailed, multicolor 3D liver models.
In addition to making a 3D model, the software can provide a map that displays the risks associated with resecting different areas around the tumors, which may help surgeons determine the optimal resection strategy, Schenk told AuntMinnieEurope.com.
Presurgical planning and risk analysis for the removal of a centrally located liver tumor. The patient-specific 3D model (top) based on CT scans (bottom) of the liver. All images courtesy of Fraunhofer MEVIS."There are studies showing the benefits of the software when comparing presurgical planning without it to presurgical planning using all of the information [provided by the software]," she said. "For individual cases, the algorithms allow [clinicians to] safely perform even delicate operations that would have been considered too risky without the software."
Liver cancer surgery
Liver tumor resection is a delicate surgical procedure that involves navigating through the organ's complex, interwoven vascular network. Even minor errors can interfere with blood flow and dramatically alter the surgical outcome, according to the researchers.
Clinicians planning the surgery typically rely on MRI or CT scans of the patient's liver for visualization, but mentally reconstructing the often-entangled liver vasculature by looking at 2D images alone can be challenging, they noted.
To bolster presurgical planning for liver surgery, Schenk and colleagues designed computer software known as HepaVision that uses AI algorithms to analyze MRI or CT scans of the liver and then takes this information to create a 3D model and an accompanying risk map.
The software performs several processes to produce the 3D models, including the following:
- Segmentation: The software segments all relevant structures -- including liver tissue, lesions, and blood vessels -- from 2D images. For example, an algorithm unites the portal vein and hepatic vein, along with their subtrees, into a single dataset separate from nonrelated structures such as tumors.
- Labeling: The software assigns different colors for each distinct structure.
- Resectioning: Resections refer to a collection of lines drawn on the surface of the liver that allow the 2D image slices to convert into a 3D plane interchangeably, Schenk said.
- Analysis: Algorithms automatically compute the distance between vessels and lesions, volumes, and areas of insufficient blood supply or drainage, among other factors. The analysis also includes a risk map, which includes spaces where vessels are extremely close to tumors. Surgeons can use this information to optimize their presurgical planning.
- Visualization: Finally, the software combines the previously segmented sections into one 3D virtual model.
By disclosing measurements of the vessels and their degree of blood supply or drainage, the risk analysis enables clinicians to determine the risk of cutting a vessel at any point of a particular tumor resection path.
Furthermore, the software proposes the best tumor resection strategy for each patient, although the final decision must be made by the surgical team, Shenk noted.
Liver transplantation
Bringing this software into the operating room, Dr. Alexander Köhn of Fraunhofer MEVIS collaborated with physicians at Yokohama National University in Japan to develop an app called Mobile Liver Explorer that allows clinicians to view presurgical planning information on an iPad for reference during a procedure.
Visualizing separate regions of the patient's liver in distinct colors in 3D and also knowing the precise distances between vessels and tumors are particularly important for maintaining liver function in donors after liver transplantation, Schenk said. The information on the iPad can provide guidance during surgery and minimize the risk of postoperative organ failure.
The Mobile Liver Explorer iPad app in use during a liver operation at Yokohama City University Hospital.The iPad app is also compatible with augmented reality. Surgeons can turn on the iPad camera and superimpose a 3D model of the patient's liver, previously made with HepaVision, directly onto the camera's view. In this way, the app reveals the position of the blood vessels and tumors beneath the liver surface.
"There is currently an ongoing study in Yokohama to show the benefit of the iPad in the operating room, looking at operating time, blood loss, and outcome in cases of Klatskin tumors," she said. "Our clinical partners in Essen and Kyoto have also derived algorithms for selecting patients for living donor liver transplantation ... based on the MEVIS software analysis."
In addition to AI algorithm and computer science expertise, communication with surgeons was important for identifying the kind of software that was needed in clinical practice, Köhn said. And continued exchange with clinical experts will help improve the software.
