Applications of Augmented and Virtual Reality in Spine Surgery

In December 2019, the FDA approved the first augmented reality (AR) guidance system for surgery, which is currently intended for spine and pelvic surgery. AR and virtual reality (VR) have shown benefits in several fields of health care, and spine surgery is soon expected to benefit from additional commercial and custom-made AR/VR systems.

Subscribe to the latest updates from Orthopaedics Advances in Motion

Thank you for subscribing!

Error: Please enter a valid email address.

Email Address Submit

In The Spine Journal, Massachusetts General Hospital researchers recently reviewed the applications and limitations of AR and VR in spine surgery, and explored what the future may hold for these emerging technologies. The authors are Hamid Ghaednia, PhD, co-director of the SORG Orthopaedic Research Group, co-director of the Center of Physical Artificial Intelligence (CPAI) and an instructor in the Orthopaedic Spine Center at Mass General, and Joseph H. Schwab MD, chief of the Orthopaedic Spine Center, director of the SORG Orthopaedic Research Group and CPAI & co-director of the Stephan L. Harris Chordoma Center at Mass General Cancer Center, and colleagues.

Descriptions and Components

Both VR and AR systems consist of three main components: the tracking system (external motion sensors or machine vision), the display (ideally, a headset or goggles, but sometimes a tablet, computer monitor or smartphone) and the computation system (usually a regular personal computer).

In health care, VR and AR systems are typically integrated with an imaging modality, such as radiography, ultrasound, CT or MRI, and use computer vision techniques and deep learning to create a link between images and patients.

Virtual reality is a completely immersive experience—the subject is placed within a fully developed, interactive virtual environment through the use of multiple sensors and controllers. In the operating room, VR must be integrated with a surgical robot.

Augmented reality allows physical interaction with the real world. For example, the newly approved image guidance system, xvision (Augmedics, Chicago, IL), determines the position of surgical tools in real-time and superimposes a virtual trajectory on the patient's CT data. 3D navigation data is projected onto the surgeon's retina using a headset.

Mixed reality allows physical interaction with virtual objects. For example, goggles can project a 3D model of the patient's spine, and the surgeon can move virtual objects such as screws to practice or plan a procedure. The two commercially available goggles are the Microsoft Hololens (Redwood, WA), which is the more powerful of the two, and Magic Leap (Ft. Lauderdale, FL), which is lighter in weight, as the visual processing module is placed on the hip.

Applications

Navigation is currently the most researched application of AR/VR in spinal surgery. This is unsurprising, considering the increasing prevalence of minimally invasive spine surgeries where navigation is more challenging and the anatomy of the posterior spinal column is not visible. AR/VR is expected to be particularly useful for placing pedicle screws accurately.

Other applications of AR/VR will include:

• Training of students, residents and fellows
• Preoperative planning, especially for less experienced surgeons, because VR and AR provide better visualization of imaging results and allow the repeated practice of technique
• Mentoring and consultation—using a combination of VR and AR, an experienced, geographically remote surgeon can overlay suggestions onto another surgeon's view
• Patient education and shared decision-making via better visualization

Limitations

Technological limitations currently constrain the health care applications of AR/VR, especially in surgery:

• Lack of physical sensation interferes with the high precision a surgeon requires; gloves that provide a sense of touch and haptic feedback suits ("Tesla suits") that simulate physical forces on the body are largely inadequate
• Blocked field of view, dizziness, headaches and blurred vision are often problems with goggles
• Learning curves are steep

Envisioning a New Future

Despite their limitations, the future of AR/VR systems in spine surgery is promising due to their unique capabilities, low cost and flexibility in integrating with other technologies. There is potential to:

• Incorporate gaming, the leading technology for AR/VR, into patient and student education and pain distraction for patients
• Combine artificial intelligence (AI), wearable sensors and gaming with AR/VR to enable the development of virtual physical rehabilitation environments where the physician will interact with the patient remotely
• Make robot-assisted surgical navigation systems faster and more precise by integrating them with AI and AR
• Combine AI, wearables and AR to provide surgeons with live feedback during procedures
• Merge AI, AR and VR to improve remote mentoring
• Integrate AR/VR with wearables, AI and surgical robotics to permit remote and semiautomatic surgeries

Collaborations between surgeons, engineers and game designers are expected to improve AR/VR technology for spine surgery profoundly within the next decade.

view original journal article Subscription may be required

Learn more about the Orthopaedic Spine Center

Learn more about the Orthopaedic Spine Surgery Fellowship at Mass General