Object-Based Geometric Distortion Metric for Viewport Rendering of 360° Images

Abstract

To visualize omnidirectional (or 360°) visual content, a sphere to plane projection is employed, that maps pixels from the observed sphere region to a 2D image, called as viewport. However, this projection introduces geometrical distortions on the rendered image, such as object shape stretching, or shearing, and bending of straight lines, which may affect the user’s quality of experience (QoE). This paper proposes an object-based quality metric to assess the subjective impact of the objects shape deformation. The metric uses semantic segmentation to identify the relevant objects in the viewport, where the stretching distortion has a higher perceptual impact, and computes the stretching distortion for each object. Two distinct approaches were exploited and evaluated: the first one, directly computes and compares object shape measures on the sphere and on the viewport; the second one is based on Tissot indicatrices, which are computed for individual objects in the viewport. The experimental results show that while the Tissot based method performs slightly better than the direct shape measurement, both approaches outperform benchmark solutions; furthermore, they are able to classify the viewport quality, with respect to quality scores obtained in a subjective crowdsourcing study, with a correct decision percentage close to 90%. Additionally, the Tissot based approach was used in a global quality metric that finds out the Pannini projection parameters that result in the least perceivable geometric distortion. It is shown that the automatically tuned Pannini projection results in viewports with a more pleasant visual quality than the considered benchmark projections.

Keywords

• Omnidirectional images
• viewport rendering
• geometric distortions
• objective quality assessment
• Pannini projection
• content-aware projection