Deep Convolutional Feature-Driven Rate Control for the HEVC Encoders

Abstract

This work proposes a rate control model based on deep convolutional features to improve the video coding performance of the HEVC encoders under the random access (RA) configuration. The proposed algorithm extracts high-level features from the original and previous coded frames using a pretrained visual geometry group (VGG-16) model by considering characteristics of a different temporal layer for the RA configuration. Subsequently, R– $\lambda $ parameters (alpha and beta), bit allocation, $\lambda $ estimation, and quantization parameter decision at frame-level are formulated by utilizing the extracted high-level features to maintain video quality and bitrate accuracy control. In addition, bit allocation at the group-of-picture (GOP)-level rate control is proposed with perceptual-based thresholding to control smooth bitrates and visual quality between adjacent GOPs. The results verify that the proposed algorithm is efficient in coding performance and bit accuracy by keeping visual quality. Compared with the existing R– $\lambda $ rate model in HM-16.20, the proposed models can achieve an average BD-rate gain of −4.39% and −8.74% in PSNR and MSSSIM metrics for the RA configuration, respectively.

Keywords

• Deep neural network
• high efficiency video coding (HEVC)
• perceptual video coding
• rate control
• video coding