Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation

Abstract

Major hurdles for Microsoft's HoloLens as a tool in medicine have been accessing tracking data, as well as a relatively high-localisation error of the displayed information; cumulatively resulting in its limited use and minimal quantification. The following work investigates the augmentation of HoloLens with the proprietary image processing SDK Vuforia, allowing integration of data from its front-facing RGB camera to provide more spatially stable holograms for neuronavigational use. Continuous camera tracking was able to maintain hologram registration with a mean perceived drift of 1.41 mm, as well as a mean sub 2-mm surface point localisation accuracy of 53%, all while allowing the researcher to walk about a test area. This represents a 68% improvement for the later and a 34% improvement for the former compared with a typical HoloLens deployment used as a control. Both represent a significant improvement on hologram stability given the current state-of-the-art, and to the best of the authors knowledge are the first example of quantified measurements when augmenting hologram stability using data from the RGB sensor.

Keywords

• holography
• medical image processing
• image registration
• cameras
• object tracking
• minimal quantification
• proprietary image processing SDK Vuforia
• spatially stable holograms
• continuous camera tracking
• hologram registration
• typical HoloLens deployment
• augmenting hologram stability
• vuforia tracking
• neuronavigation
• tracking data
• high-localisation error
• Microsoft HoloLens augmentation
• surface point localisation accuracy
• front-facing RGB camera
• mean perceived drift
• size 1.41 mm