Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

Yannik Kalbas; Hoijoon Jung; John Ricklin; Ge Jin; Mingjian Li; Thomas Rauer; Shervin Dehghani; Nassir Navab; Jinman Kim; Hans-Christoph Pape; Sandro-Michael Heining

doi:10.3390/jimaging9030056

Journal of Imaging (Feb 2023)

Remote Interactive Surgery Platform (RISP): Proof of Concept for an Augmented-Reality-Based Platform for Surgical Telementoring

Yannik Kalbas,
Hoijoon Jung,
John Ricklin,
Ge Jin,
Mingjian Li,
Thomas Rauer,
Shervin Dehghani,
Nassir Navab,
Jinman Kim,
Hans-Christoph Pape,
Sandro-Michael Heining

Affiliations

Yannik Kalbas: Department of Trauma Surgery and Harald-Tscherne Laboratory, University Hospital Zurich, Ramistr. 100, 8091 Zurich, Switzerland
Hoijoon Jung: School of Computer Science, University of Sydney, Camperdown, NSW 2006, Australia
John Ricklin: Department of Trauma Surgery and Harald-Tscherne Laboratory, University Hospital Zurich, Ramistr. 100, 8091 Zurich, Switzerland
Ge Jin: School of Computer Science, University of Sydney, Camperdown, NSW 2006, Australia
Mingjian Li: School of Computer Science, University of Sydney, Camperdown, NSW 2006, Australia
Thomas Rauer: Department of Trauma Surgery and Harald-Tscherne Laboratory, University Hospital Zurich, Ramistr. 100, 8091 Zurich, Switzerland
Shervin Dehghani: TUM School of Computation, Information and Technology, Technische Universität München, 85748 München, Germany
Nassir Navab: TUM School of Computation, Information and Technology, Technische Universität München, 85748 München, Germany
Jinman Kim: School of Computer Science, University of Sydney, Camperdown, NSW 2006, Australia
Hans-Christoph Pape: Department of Trauma Surgery and Harald-Tscherne Laboratory, University Hospital Zurich, Ramistr. 100, 8091 Zurich, Switzerland
Sandro-Michael Heining: Department of Trauma Surgery and Harald-Tscherne Laboratory, University Hospital Zurich, Ramistr. 100, 8091 Zurich, Switzerland

DOI: https://doi.org/10.3390/jimaging9030056
Journal volume & issue: Vol. 9, no. 3
p. 56

Abstract

Read online

The “Remote Interactive Surgery Platform” (RISP) is an augmented reality (AR)-based platform for surgical telementoring. It builds upon recent advances of mixed reality head-mounted displays (MR-HMD) and associated immersive visualization technologies to assist the surgeon during an operation. It enables an interactive, real-time collaboration with a remote consultant by sharing the operating surgeon’s field of view through the Microsoft (MS) HoloLens2 (HL2). Development of the RISP started during the Medical Augmented Reality Summer School 2021 and is currently still ongoing. It currently includes features such as three-dimensional annotations, bidirectional voice communication and interactive windows to display radiographs within the sterile field. This manuscript provides an overview of the RISP and preliminary results regarding its annotation accuracy and user experience measured with ten participants.

Published in Journal of Imaging

ISSN: 2313-433X (Online)
Publisher: MDPI AG
Country of publisher: Switzerland
LCC subjects: Technology: Photography; Medicine: Medicine (General): Computer applications to medicine. Medical informatics; Science: Mathematics: Instruments and machines: Electronic computers. Computer science
Website: http://www.mdpi.com/journal/jimaging

About the journal

Abstract

Keywords