Health-5G: A Mixed Reality-Based System for Remote Medical Assistance in Emergency Situations

Abstract

Mixed reality is the combination of virtual and augmented reality to interactively and believably merge physical and computer-generated environments. This paper discusses the design of Health-5G, a scalable mixed reality-based system that facilitates and supports emergency response by medical emergency teams. Health-5G is supported by a distributed architecture divided into four interrelated applications responsible for advanced computer-human interaction, effective real-time videoconference, medical device integration, and communication infrastructure, respectively. The mixed reality layer is provided by the headset Microsoft Hololens 2™. Health-5G is based on scenarios in which emergency personnel wear mixed reality glasses that can transmit audio, video, and data streams bidirectionally over a 5G network to medical specialists stationed in a hospital at any distance. Thanks to Health-5G, the specialist will be able to access the emergency team’s point of view at any time and provide verbal and visual instructions, including gestures and positioning of graphical markers in 3D space. In this way, emergency personnel can provide the best possible care to the patient without having to wait for them to arrive at the hospital, saving a lot of time in scenarios where every second can make a difference. Health-5G also addresses the integration of medical devices and the collection of the patient’s medical data in a scalable way through optical character recognition. A case study is discussed where Health-5G is used to attend a patient in the street suffering from syncope due to third-degree atrioventricular block. Latency and performance tests over a 5G network are also discussed. To the best of our knowledge, there is no comprehensive solution in the literature that provides all the capabilities offered by Health-5G in terms of functionality and advanced interaction mechanisms within the context of remote, immersive support in emergency situations.

Keywords

• Mixed reality
• teleassistance
• 5G
• visualization