Researchers from Spain have developed a smartphone app for visualizing patient-specific 3D models from imaging data. They have outlined step by step the process through which clinicians and other users can create these models on their own.
The researchers, led by Javier Pascau, PhD, from Universidad Carlos III de Madrid, describe how clinicians can create patient-specific virtual 3D models based on medical images acquired using any modality and then superimpose these models onto a real-world environment using 3D-printed markers and a smartphone camera.
Clinicians can use the 3D models for a number of applications, from facilitating patient-to-physician communication to guiding target localization during surgical procedures, the authors noted. The combination of augmented reality (AR) and 3D printing technology are vital components of the technique (Journal of Visualized Experiments, 2 January 2020).
Visualization of a patient's distal leg sarcoma as a 3D model on a smartphone. Image courtesy of Pascau et al."This methodology does not require broad knowledge of medical imaging or software development, does not depend on complex hardware and expensive software, and can be implemented over a short time period," they noted. "It is expected that this method will help accelerate the adoption of AR and 3D printing technologies by medical professionals."
3D models on smartphones
Recent studies have shown the capacity of AR and 3D printing technologies to improve orientation and spatial skills for medical procedures as well as enhance patient engagement and education.
Yet despite the demonstrated benefits of these technologies, their adoption by physicians remains rather limited, the authors noted. The integration of AR and 3D printing into clinical practice often requires extensive knowledge of engineering and software development and also brings with it significant time and cost considerations.
In light of these barriers, Pascau and colleagues developed a method that makes the use of AR and 3D printing in clinical settings more accessible to users with minimal technical knowledge of the technologies. Their method consists of several parts summarized in the following steps:
- Setup: The protocol requires users to download and install multiple open-source software, including 3D Slicer, Meshmixer, and Unity, and software modules from the GitHub data repository, all of which are freely available online.
- Creating the 3D models: Clinicians can create virtual 3D models from patient imaging data acquired using any standard modality by inputting the medical image file into open-source segmentation software, segmenting target anatomy and pathology, and exporting the files as 3D models. The group's article details the precise steps involved in this process.
- Model positioning: The 3D models need to be aligned with special markers to be able to see them with AR technology. The software allows for automatic registration of the models to ensure proper positioning, orientation, and scaling. Users can further position the models as they see fit following step-by-step instructions that are posted directly online in the download section for the segmentation software module OpenARHealth.
- 3D printing: The group's method also involves 3D printing the virtual 3D models and the associated AR markers. The mobility of the markers enables users to see 3D models on their phones wherever they place the markers.
- Smartphone app: Pascau and colleagues also describe exactly how clinicians can develop entirely on their own the app to view 3D models on an Android or iOS smartphone. The process entails no more than a dozen simple steps.
- Visualization: Finally, users will be able to open their self-made app, choose the patient-specific 3D models they would like to view on a smartphone, and alter the positioning of the models as needed. The app displays a selected 3D model on the smartphone screen as soon as the camera recognizes the corresponding 3D-printed marker.
The researchers' method allows any clinician with a smartphone to visualize patient imaging data in 3D in any number of real-world settings for a relatively low cost. To test the feasibility of their technique, the researchers applied it to a clinical case of a patient who underwent a CT exam to evaluate a distal leg sarcoma.
The combination of AR and 3D printing can "become a powerful solution for physicians ... which can help improve preoperative planning and clinical interventions," the authors wrote.
For clinicians interested in getting started, the researchers have produced and posted online a brief video explaining their process for creating and visualizing 3D models on a smartphone.
