“Wall Hack AR”: AR See-through system with LiDAR and VPS

Abstract

We propose a method to realize the experience of seeing through an occluding wall by using a 3D scanner and an AR (Augmented Reality) device. This system uses the LiDAR (Light Detection And Ranging) scanner of the Apple iPad Pro as a 3D scanner, sending scanned spatial mesh data to the AR device of the experiencer in real-time. By using a VPS (Visual Positioning System) to also share the actual spatial position of the mesh, a receiving AR device can also draw the scan data registered at the correct position in the actual space. Placing the LiDAR scanner behind an occlusion allows an experience as if the user were seeing through a wall.