Using an augmented reality (AR) headset to view CT angiography (CTA) images allowed researchers from the U.K. to improve the accuracy and speed of reconstructive surgery in the leg for a series of cases detailed in an article published online on 31 January in European Radiology Experimental.
The team of surgeons and radiologists from Imperial College London modified CTA scans of the lower limb through various computer software. By wearing an augmented reality gadget, they were able to visualize medical images directly on the patient's leg and, what's more, manipulate this virtual anatomy with hand gestures during surgery (Eur Radiol Exp, 31 January 2018).
Incorporating augmented reality in the operating room facilitated the identification, dissection, and attachment of blood vessels during leg surgery and ultimately reduced surgical time and morbidity, according to the authors.
"There are a number of areas we would like to explore, and further improvements are needed, but the small case series has shown that for reconstructive surgery, this [augmented reality technique] seems to be a valuable tool in the operating theater," lead author Dr. Philip Pratt noted in a statement from the university.
Pioneering work
Traumatic injuries involving tissue damage or open wounds often require reconstructive surgery, for which flaps of tissue -- including skin and blood vessels -- are extracted from an unaffected area and integrated into the wounded area for proper healing and closure.
Presently, the gold standard for connecting the blood vessels of undamaged tissue to vessels around an injury relies on handheld ultrasound scanning to identify the location of the vessels as blood flows through them. Ultrasound scans can help surgeons estimate the location of blood vessels near the wound that they can eventually attach to the "new" tissue. But ultrasound is limited in its visualization of the vasculature and is also restricted to preoperative use for this application.
Aiming to resolve this issue, Pratt and colleagues combined the image quality of CT to the potential intraoperative application of augmented reality in a new approach to reconstructive flap surgery.
They began by acquiring contrast-enhanced CTA scans of patients in need of reconstructive surgery in the lower extremity. Scans of each patient then went through several rounds of image processing: segmentation into skin, bone, muscle, fat, and vascular models (Vitrea 6.7, Vital Images); refinement via an open-source software known as ITK-SNAP 3.6; and smoothing with the open-source software MeshLab 1.3.3.
The group then input the edited images into a custom-developed application written within a program (Unity 2017.1, Unity Technologies) that is compatible with an augmented reality device (HoloLens, Microsoft). After launching the application, they only needed to put on the HoloLens augmented reality headset to be able to see holographic overlays of the patients' internal anatomy directly on their legs.
Surgeon's view of lower limb reconstructive surgery using a Microsoft HoloLens headset. Image courtesy of Dr. Philip Pratt et al."With the [augmented reality device], you look at the leg and essentially see inside of it," Pratt said. "You see the bones, the course of the blood vessels, and can identify exactly where the targets are located. Augmented reality offers a new way to find these blood vessels under the skin accurately and quickly by overlaying scan images onto the patient during the operation."
'Exciting possibilities'
Beyond seeing a patient's anatomy in vivo, clinicians wearing a HoloLens headset can manipulate the augmented reality images by signaling with their hands. For example, operating surgeons can align the virtual images to the current patient anatomy, adjusting for any changes that may have occurred to key landmarks since the time of initial image acquisition.
Whereas in the past, surgeons had to rely on rough measurements of anatomical landmarks taken from CT reconstructions to guide procedures, they can now pinpoint where the blood vessels are in 3D space and follow virtual arrows throughout the operation, said Dr. Dimitri Amiras, consultant radiologist at Imperial College Healthcare National Health Service Trust.
Surgeons led by Dr. Jon Simmons at St. Mary's Hospital used the technology to perform reconstructive flap surgery on the lower limbs of five patients with severe trauma to the area. A few of the cases involved gaping open fractures, and the patients' ages ranged from 41 to 85 years. Utilizing the HoloLens was more reliable and less time-consuming than the ultrasound technique of locating blood vessels, according to the surgical team.
"The application of AR technology in the operating theater has some really exciting possibilities," Simmons said. "It could help to simplify and improve the accuracy of some elements of reconstructive procedures. While the technology can't replace the skill and experience of the clinical team, it could potentially help to reduce the time a patient spends under anesthetic and reduce the margin for error."
Among limitations noted by the authors are potential errors during modeling of the CT scans, minor misalignment of the virtual images, and requiring a technical assistant to assist in launching the augmented reality application and approximate image positioning.
They plan to begin automating some of the data processing steps to speed up preparation of the images. They also believe the augmented reality technique could be applied to other surgeries that depend on transferring tissue flaps, such as breast reconstruction after mastectomy.
"In the future, we hope to automate the process further," Pratt said. "We can use software to improve the alignment and will attach markers to the patient when they have the scan, with the same markers present during the operation to use as additional points of reference."
