Velocity- and Error-Aware Switching of Motion Prediction Models for Cloud Virtual Reality

Abstract

Offloading virtual reality (VR) computations to a cloud computing entity can enable support for VR services on low-end user devices but may result in increased latency, which will lead to mismatch between the user’s viewport and the received VR image, thus inducing motion sickness. Predicting future motion and rendering future images accordingly is a promising solution to the latency problem. In this paper, we develop velocity- and error-aware model switching schemes applicable to a wide range of existing motion prediction models. First, we consider the chattering problem of machine learning (ML)-based prediction models and the relationship between the velocity and the prediction error gap between an ML model and the case of no prediction (NOP). Accordingly, we propose a velocity-aware switching (VAS) scheme that combines the outputs from the ML model and the NOP case via a weight determined by the head motion velocity. Next, we develop an ensemble method combining a set of outputs from VAS and other models, called error-aware switching (EAS). EAS switches between model outputs based on the error statistics of those outputs under the parallel execution of multiple models, including VAS models. For EAS, schemes for both hard switching and soft integration of the model outputs are proposed. We evaluate the proposed schemes based on real VR motion traces for diverse ML-based prediction models.

Keywords

• Virtual reality
• VR
• cloud VR
• motion prediction
• machine learning
• ensemble