Sub‐bitstream packing based lightweight tiled streaming for 6 degree of freedom immersive video

Abstract

Abstract The MPEG immersive video (MIV) has been designed to remove the redundancies of multi‐view videos and merge the residuals into two textures and two geometry videos; therefore, four bitstreams and video decoders are required to provide six degrees of freedom (6DoF) in virtual reality (VR) in MIV. This letter proposes a sub‐bitstream packing method and realisation in tiled streaming for MIV to provide a single bitstream for legacy systems that have a single decoder without an increase in the server‐side storage. The proposed method divided the input videos into tiles in the compressed‐domain to accomplish two goals: (i) generate a merged bitstream using the existing four bitstreams for legacy systems and (ii) improve the quality of an immersive video by disabling the deblocking filter. For a player with one or few numbers of decoders, the extracted tile sub‐streams were merged by the proposed four specific mixed‐quality tiling and packing methods using a motion‐constrained tile set (MCTS), while complying with the high‐efficiency video coding (HEVC) standard. Experimental results demonstrated that the proposed sub‐stream packing method required a single decoder and exhibited 17.40% decoding time and 0.81% bjontegaard‐delta rate (BD‐rate) savings on Y‐PSNR saving compared to the non‐tiled streaming method.

Keywords

• Image and video coding
• Computer vision and image processing techniques
• Video signal processing
• Virtual reality
• Combinatorial mathematics
• Combinatorial mathematics