Adaptive Patch-Wise Depth Range Linear Scaling Method for MPEG Immersive Video Coding

Abstract

The Moving Picture Experts Group (MPEG) is responsible for standardizing MPEG immersive video (MIV) for immersive video coding and is involved in research and development focusing on providing six degrees of freedom through a reference software known as the test model for immersive video. To efficiently compress and transmit multiview videos with texture and depth pairings, the encoder part of the MIV codec framework reduces the pixel rate by removing redundancy between views and densely packing the remaining regions into an atlas as patches. The decoder part reconstructs multiview videos from the transmitted atlas to synthesize and render arbitrary viewports, and the depth information has a significant impact on the quality of the rendered viewport. However, the existing method of handling depth values in the MIV codec fails to adequately address the information loss that occurs during quantization or transmission. To preserve and transmit depth information more accurately, we propose a method for expanding the depth dynamic range using min–max linear scaling on a patch-by-patch basis. In addition, we efficiently encode the per-patch minimum and maximum values of depth required by the decoder to recover the original depth values and include them in the metadata. The experimental results indicate that for computer-generated sequences, the proposed method provides PSNR-based Bjøntegaard delta-rate gains of 9.1% and 3.3% in the end-to-end performance for high- and low-bitrate cases, respectively. In addition, subjective quality improvements are observed by reducing the artifacts that primarily occur at the object boundaries in the rendered viewport.

Keywords

• Depth scaling
• immersive video
• MPEG immersive video
• multiview video
• virtual reality
• 6DoF video