3D Panoramic Virtual Reality Video Quality Assessment Based on 3D Convolutional Neural Networks

Abstract

Virtual reality (VR), a new type of simulation and interaction technology, has aroused widespread attention and research interest. It is necessary to evaluate the VR quality and provide a standard for the rapidly developing technology. To the best of our knowledge, a few researchers have built benchmark databases and designed related algorithms, which has hindered the further development of the VR technology. In this paper, a free available data set (VRQ-TJU) for VR quality assessment is proposed with subjective scores for each sample data. The validity for the designed database has been proved based on the traditional multimedia quality assessment metrics. In addition, an end-to-end 3-D convolutional neural network is introduced to predict the VR video quality without a referenced VR video. This method can extract spatiotemporal features and does not require using hand-crafted features. At the same time, a new score fusion strategy is designed based on the characteristics of the VR video projection process. Taking the pre-processed VR video patches as input, the network captures local spatiotemporal features and gets the score of every patch. Then, the new quality score fusion strategy is applied to get the final score. Such approach shows advanced performance on this database.

Keywords

• Virtual reality quality assessment
• benchmark database
• 3D convolutional neural networks
• spatiotemporal features
• quality score fusion strategy