Dutch researchers have used augmented reality to help pinpoint brain tumors before surgically removing them. They found the technique was nearly as accurate as standard neuronavigation, and it has helped improve understanding of the anatomy and efficiency during a procedure.
The group, led by first author Dr. Fatih Incekara from Erasmus University Medical Center in Rotterdam, converted MRI scans of brain tumor patients into holograms and then overlaid these holograms directly onto the head of each patient using a type of augmented reality technology known as mixed reality. This allowed clinicians to visualize the brain and tumor of each patient right before surgery (World Neurosurg, October 2018, Vol. 118, pp. e422-e427).
Mixed reality view of a hologram of a brain tumor directly on the patient's head. The tumor (red) is outlined. Image courtesy of Dr. Fatih Incekara."This new technology proved to be pretty accurate when compared with the neuronavigation system," Incekara told AuntMinnieEurope.com. "And surgeons reported an improved focus during surgery."
Tumor localization
In recent decades, image-guided navigation technologies have proved highly practical and effective for use in complex neurosurgeries such as brain tumor removal, the authors noted. Current neuronavigation methods using these tools, however, are impeded by long setup times and require clinicians to repeatedly shift their gaze back and forth between 2D images on a screen and the patient.
Seeking to find a more intuitive way to plan tumor surgery, Incekara and colleagues turned to mixed reality technology -- a variation of augmented reality that allows users to see and interact with virtual objects and their actual surroundings. They tested the feasibility of this approach in presurgical planning for the removal of brain tumors from 25 patients at their institution.
The group began by acquiring and then segmenting brain MRI scans of the patients. Next, they reconstructed 3D virtual models of the brain tumors with open-source software (Meshmixer, Autodesk) and used 3D-modeling software (Verto Studio) to make these virtual models compatible for use in a mixed reality device (Hololens, Microsoft).
Wearing a Hololens headset, the neurosurgeons were able to merge the hologram of the patient's brain and tumor with the physical head of the patient. This enabled them to determine the location of tumors and then mark the center and border of each tumor directly on the patient's head. For comparison, the surgical team marked the center and border of the tumors on the patient's head again, but this time using the standard neuronavigation method. They eventually removed the brain tumors based on measurements from the standard method.
Overall, the holograms of the tumor aligned well with the borders of the actual tumor. The clinicians were able to locate tumors using the mixed reality technique as precisely as they did with the conventional neuronavigation method in nine of the 25 patients (36%). The median deviation between the two techniques was 0.4 cm for the rest of the cases.
Furthermore, the surgeons claimed that using the mixed reality device during the operation enhanced their ergonomics, their understanding of the relationship between the tumor and the brain, and their attention and focus on the patient.
Future of surgery
The authors acknowledged there were several limitations with their technique: the slightly decreased localization accuracy (0.4-cm deviation) and longer time required for image registration (45 more seconds). However small, the slight deviation between the mixed reality technique and conventional neuronavigation suggests the new method may be suitable for preoperative planning, but not yet for intraoperative use.
Still, the surgeons' measurements using the mixed reality device became increasingly more accurate over time -- starting with a median deviation of 0.6 cm for the first 12 cases and ending with a median deviation of 0.3 cm for the last 13 cases.
To improve upon these aspects, the researchers are developing a coordination-based image registration process that may improve the spatial accuracy and registration time of their technique.
Additional research using mixed reality technology will be necessary to verify the applicability of this and similar methods for planning neurosurgeries, noted Incekara, who is currently studying for a PhD. Introducing other features such as overlaying patient history and imaging data could also prove beneficial.
"I think the future of surgery is augmented reality navigated and data (imaging and AI) driven," he said. "We are closing the era of looking at a 2D screen only to struggle to interpret and translate that data to the patient in the operating room."
